2023-03-08 19:53:32,679 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor timeout: 13 mins 2023-03-08 19:53:32,969 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/36526ac9-e9a0-8ea5-8063-2f26c591d24b 2023-03-08 19:53:32,981 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=1, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-03-08 19:53:32,982 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/36526ac9-e9a0-8ea5-8063-2f26c591d24b/cluster_63dc3982-03ab-365c-777c-42eb5a69ef67, deleteOnExit=true 2023-03-08 19:53:32,982 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-03-08 19:53:32,983 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/36526ac9-e9a0-8ea5-8063-2f26c591d24b/test.cache.data in system properties and HBase conf 2023-03-08 19:53:32,983 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/36526ac9-e9a0-8ea5-8063-2f26c591d24b/hadoop.tmp.dir in system properties and HBase conf 2023-03-08 19:53:32,983 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/36526ac9-e9a0-8ea5-8063-2f26c591d24b/hadoop.log.dir in system properties and HBase conf 2023-03-08 19:53:32,984 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/36526ac9-e9a0-8ea5-8063-2f26c591d24b/mapreduce.cluster.local.dir in system properties and HBase conf 2023-03-08 19:53:32,984 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/36526ac9-e9a0-8ea5-8063-2f26c591d24b/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-03-08 19:53:32,984 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-03-08 19:53:33,099 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-03-08 19:53:33,504 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-03-08 19:53:33,509 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/36526ac9-e9a0-8ea5-8063-2f26c591d24b/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-03-08 19:53:33,509 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/36526ac9-e9a0-8ea5-8063-2f26c591d24b/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-03-08 19:53:33,509 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/36526ac9-e9a0-8ea5-8063-2f26c591d24b/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-03-08 19:53:33,510 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/36526ac9-e9a0-8ea5-8063-2f26c591d24b/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-03-08 19:53:33,510 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/36526ac9-e9a0-8ea5-8063-2f26c591d24b/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-03-08 19:53:33,510 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/36526ac9-e9a0-8ea5-8063-2f26c591d24b/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-03-08 19:53:33,510 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/36526ac9-e9a0-8ea5-8063-2f26c591d24b/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-03-08 19:53:33,511 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/36526ac9-e9a0-8ea5-8063-2f26c591d24b/dfs.journalnode.edits.dir in system properties and HBase conf 2023-03-08 19:53:33,511 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/36526ac9-e9a0-8ea5-8063-2f26c591d24b/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-03-08 19:53:33,512 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/36526ac9-e9a0-8ea5-8063-2f26c591d24b/nfs.dump.dir in system properties and HBase conf 2023-03-08 19:53:33,512 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/36526ac9-e9a0-8ea5-8063-2f26c591d24b/java.io.tmpdir in system properties and HBase conf 2023-03-08 19:53:33,512 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/36526ac9-e9a0-8ea5-8063-2f26c591d24b/dfs.journalnode.edits.dir in system properties and HBase conf 2023-03-08 19:53:33,513 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/36526ac9-e9a0-8ea5-8063-2f26c591d24b/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-03-08 19:53:33,513 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/36526ac9-e9a0-8ea5-8063-2f26c591d24b/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-03-08 19:53:34,106 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-03-08 19:53:34,110 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-03-08 19:53:34,395 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-03-08 19:53:34,562 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-03-08 19:53:34,577 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-03-08 19:53:34,615 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-03-08 19:53:34,650 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/36526ac9-e9a0-8ea5-8063-2f26c591d24b/java.io.tmpdir/Jetty_localhost_33657_hdfs____75u0te/webapp 2023-03-08 19:53:34,824 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33657 2023-03-08 19:53:34,838 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-03-08 19:53:34,838 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-03-08 19:53:35,314 WARN [Listener at localhost/42127] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-03-08 19:53:35,439 WARN [Listener at localhost/42127] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-03-08 19:53:35,465 WARN [Listener at localhost/42127] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-03-08 19:53:35,473 INFO [Listener at localhost/42127] log.Slf4jLog(67): jetty-6.1.26 2023-03-08 19:53:35,478 INFO [Listener at localhost/42127] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/36526ac9-e9a0-8ea5-8063-2f26c591d24b/java.io.tmpdir/Jetty_localhost_46713_datanode____.rz3lxe/webapp 2023-03-08 19:53:35,584 INFO [Listener at localhost/42127] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:46713 2023-03-08 19:53:35,982 WARN [Listener at localhost/34049] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-03-08 19:53:36,568 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xc0748f164334f36b: Processing first storage report for DS-7ccd58a4-abcb-43bf-82b0-b526b26a4701 from datanode cf18edd4-f033-4bf5-a039-714536ab9194 2023-03-08 19:53:36,570 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xc0748f164334f36b: from storage DS-7ccd58a4-abcb-43bf-82b0-b526b26a4701 node DatanodeRegistration(127.0.0.1:41113, datanodeUuid=cf18edd4-f033-4bf5-a039-714536ab9194, infoPort=39949, infoSecurePort=0, ipcPort=34049, storageInfo=lv=-57;cid=testClusterID;nsid=1057443256;c=1678305214181), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-03-08 19:53:36,570 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xc0748f164334f36b: Processing first storage report for DS-4b98bc13-ddfd-4379-9c28-6baf8b1056a5 from datanode cf18edd4-f033-4bf5-a039-714536ab9194 2023-03-08 19:53:36,571 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xc0748f164334f36b: from storage DS-4b98bc13-ddfd-4379-9c28-6baf8b1056a5 node DatanodeRegistration(127.0.0.1:41113, datanodeUuid=cf18edd4-f033-4bf5-a039-714536ab9194, infoPort=39949, infoSecurePort=0, ipcPort=34049, storageInfo=lv=-57;cid=testClusterID;nsid=1057443256;c=1678305214181), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-03-08 19:53:36,591 DEBUG [Listener at localhost/34049] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/36526ac9-e9a0-8ea5-8063-2f26c591d24b 2023-03-08 19:53:36,668 INFO [Listener at localhost/34049] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/36526ac9-e9a0-8ea5-8063-2f26c591d24b/cluster_63dc3982-03ab-365c-777c-42eb5a69ef67/zookeeper_0, clientPort=56751, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/36526ac9-e9a0-8ea5-8063-2f26c591d24b/cluster_63dc3982-03ab-365c-777c-42eb5a69ef67/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/36526ac9-e9a0-8ea5-8063-2f26c591d24b/cluster_63dc3982-03ab-365c-777c-42eb5a69ef67/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-03-08 19:53:36,683 INFO [Listener at localhost/34049] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=56751 2023-03-08 19:53:36,694 INFO [Listener at localhost/34049] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-08 19:53:36,696 INFO [Listener at localhost/34049] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-08 19:53:37,368 INFO [Listener at localhost/34049] util.FSUtils(471): Created version file at hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488 with version=8 2023-03-08 19:53:37,368 INFO [Listener at localhost/34049] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/hbase-staging 2023-03-08 19:53:37,665 INFO [Listener at localhost/34049] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-03-08 19:53:38,128 INFO [Listener at localhost/34049] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=3 2023-03-08 19:53:38,160 INFO [Listener at localhost/34049] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-08 19:53:38,161 INFO [Listener at localhost/34049] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-03-08 19:53:38,161 INFO [Listener at localhost/34049] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-03-08 19:53:38,161 INFO [Listener at localhost/34049] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-08 19:53:38,162 INFO [Listener at localhost/34049] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-03-08 19:53:38,298 INFO [Listener at localhost/34049] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-03-08 19:53:38,376 DEBUG [Listener at localhost/34049] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-03-08 19:53:38,481 INFO [Listener at localhost/34049] ipc.NettyRpcServer(120): Bind to /172.31.14.131:39989 2023-03-08 19:53:38,492 INFO [Listener at localhost/34049] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-08 19:53:38,494 INFO [Listener at localhost/34049] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-08 19:53:38,515 INFO [Listener at localhost/34049] zookeeper.RecoverableZooKeeper(93): Process identifier=master:39989 connecting to ZooKeeper ensemble=127.0.0.1:56751 2023-03-08 19:53:38,556 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:399890x0, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-03-08 19:53:38,559 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:39989-0x100c5b1324f0000 connected 2023-03-08 19:53:38,587 DEBUG [Listener at localhost/34049] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-03-08 19:53:38,587 DEBUG [Listener at localhost/34049] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-08 19:53:38,591 DEBUG [Listener at localhost/34049] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-03-08 19:53:38,599 DEBUG [Listener at localhost/34049] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=39989 2023-03-08 19:53:38,599 DEBUG [Listener at localhost/34049] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=39989 2023-03-08 19:53:38,600 DEBUG [Listener at localhost/34049] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=39989 2023-03-08 19:53:38,604 DEBUG [Listener at localhost/34049] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=39989 2023-03-08 19:53:38,604 DEBUG [Listener at localhost/34049] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=39989 2023-03-08 19:53:38,610 INFO [Listener at localhost/34049] master.HMaster(444): hbase.rootdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488, hbase.cluster.distributed=false 2023-03-08 19:53:38,680 INFO [Listener at localhost/34049] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=3 2023-03-08 19:53:38,681 INFO [Listener at localhost/34049] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-08 19:53:38,681 INFO [Listener at localhost/34049] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-03-08 19:53:38,681 INFO [Listener at localhost/34049] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-03-08 19:53:38,681 INFO [Listener at localhost/34049] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-08 19:53:38,681 INFO [Listener at localhost/34049] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-03-08 19:53:38,686 INFO [Listener at localhost/34049] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-03-08 19:53:38,689 INFO [Listener at localhost/34049] ipc.NettyRpcServer(120): Bind to /172.31.14.131:33367 2023-03-08 19:53:38,692 INFO [Listener at localhost/34049] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-03-08 19:53:38,699 DEBUG [Listener at localhost/34049] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-03-08 19:53:38,700 INFO [Listener at localhost/34049] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-08 19:53:38,703 INFO [Listener at localhost/34049] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-08 19:53:38,705 INFO [Listener at localhost/34049] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:33367 connecting to ZooKeeper ensemble=127.0.0.1:56751 2023-03-08 19:53:38,709 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:333670x0, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-03-08 19:53:38,710 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:33367-0x100c5b1324f0001 connected 2023-03-08 19:53:38,710 DEBUG [Listener at localhost/34049] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-03-08 19:53:38,712 DEBUG [Listener at localhost/34049] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-08 19:53:38,713 DEBUG [Listener at localhost/34049] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-03-08 19:53:38,713 DEBUG [Listener at localhost/34049] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33367 2023-03-08 19:53:38,713 DEBUG [Listener at localhost/34049] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33367 2023-03-08 19:53:38,719 DEBUG [Listener at localhost/34049] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33367 2023-03-08 19:53:38,719 DEBUG [Listener at localhost/34049] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33367 2023-03-08 19:53:38,720 DEBUG [Listener at localhost/34049] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33367 2023-03-08 19:53:38,722 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,39989,1678305217508 2023-03-08 19:53:38,731 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-03-08 19:53:38,732 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,39989,1678305217508 2023-03-08 19:53:38,753 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-03-08 19:53:38,753 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-03-08 19:53:38,753 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-08 19:53:38,754 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-03-08 19:53:38,755 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,39989,1678305217508 from backup master directory 2023-03-08 19:53:38,755 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-03-08 19:53:38,759 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,39989,1678305217508 2023-03-08 19:53:38,759 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-03-08 19:53:38,760 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-03-08 19:53:38,760 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,39989,1678305217508 2023-03-08 19:53:38,763 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-03-08 19:53:38,765 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-03-08 19:53:39,248 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/hbase.id with ID: 6bd87d59-b210-4f5a-a6f8-92e484ffe565 2023-03-08 19:53:39,308 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-08 19:53:39,323 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-08 19:53:39,367 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x7337cf50 to 127.0.0.1:56751 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 19:53:39,403 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7631c68f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 19:53:39,428 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-03-08 19:53:39,430 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-03-08 19:53:39,447 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-03-08 19:53:39,447 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-03-08 19:53:39,449 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-03-08 19:53:39,453 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-03-08 19:53:39,454 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-03-08 19:53:39,490 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/MasterData/data/master/store-tmp 2023-03-08 19:53:39,922 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:53:39,922 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-03-08 19:53:39,922 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-08 19:53:39,922 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-08 19:53:39,922 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-03-08 19:53:39,922 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-08 19:53:39,922 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-08 19:53:39,923 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-03-08 19:53:39,924 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/MasterData/WALs/jenkins-hbase4.apache.org,39989,1678305217508 2023-03-08 19:53:39,946 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C39989%2C1678305217508, suffix=, logDir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/MasterData/WALs/jenkins-hbase4.apache.org,39989,1678305217508, archiveDir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/MasterData/oldWALs, maxLogs=10 2023-03-08 19:53:40,004 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41113,DS-7ccd58a4-abcb-43bf-82b0-b526b26a4701,DISK] 2023-03-08 19:53:40,013 DEBUG [RS-EventLoopGroup-3-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-03-08 19:53:40,080 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/MasterData/WALs/jenkins-hbase4.apache.org,39989,1678305217508/jenkins-hbase4.apache.org%2C39989%2C1678305217508.1678305219958 2023-03-08 19:53:40,081 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41113,DS-7ccd58a4-abcb-43bf-82b0-b526b26a4701,DISK]] 2023-03-08 19:53:40,082 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-03-08 19:53:40,083 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:53:40,087 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-03-08 19:53:40,088 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-03-08 19:53:40,167 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-03-08 19:53:40,184 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-03-08 19:53:40,214 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-03-08 19:53:40,228 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:53:40,235 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-03-08 19:53:40,237 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-03-08 19:53:40,255 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-03-08 19:53:40,260 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 19:53:40,261 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70127618, jitterRate=0.04498293995857239}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 19:53:40,261 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-03-08 19:53:40,263 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-03-08 19:53:40,288 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-03-08 19:53:40,288 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-03-08 19:53:40,290 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-03-08 19:53:40,292 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-03-08 19:53:40,327 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 34 msec 2023-03-08 19:53:40,328 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-03-08 19:53:40,360 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-03-08 19:53:40,366 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-03-08 19:53:40,396 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-03-08 19:53:40,402 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-03-08 19:53:40,404 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-03-08 19:53:40,411 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-03-08 19:53:40,416 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-03-08 19:53:40,420 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-08 19:53:40,422 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-03-08 19:53:40,422 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-03-08 19:53:40,437 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-03-08 19:53:40,442 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-03-08 19:53:40,442 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-03-08 19:53:40,442 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-08 19:53:40,443 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,39989,1678305217508, sessionid=0x100c5b1324f0000, setting cluster-up flag (Was=false) 2023-03-08 19:53:40,461 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-08 19:53:40,468 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-03-08 19:53:40,470 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,39989,1678305217508 2023-03-08 19:53:40,475 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-08 19:53:40,482 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-03-08 19:53:40,483 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,39989,1678305217508 2023-03-08 19:53:40,486 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.hbase-snapshot/.tmp 2023-03-08 19:53:40,525 INFO [RS:0;jenkins-hbase4:33367] regionserver.HRegionServer(952): ClusterId : 6bd87d59-b210-4f5a-a6f8-92e484ffe565 2023-03-08 19:53:40,532 DEBUG [RS:0;jenkins-hbase4:33367] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-03-08 19:53:40,542 DEBUG [RS:0;jenkins-hbase4:33367] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-03-08 19:53:40,542 DEBUG [RS:0;jenkins-hbase4:33367] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-03-08 19:53:40,550 DEBUG [RS:0;jenkins-hbase4:33367] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-03-08 19:53:40,551 DEBUG [RS:0;jenkins-hbase4:33367] zookeeper.ReadOnlyZKClient(139): Connect 0x1e1f4632 to 127.0.0.1:56751 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 19:53:40,569 DEBUG [RS:0;jenkins-hbase4:33367] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@10d8bb81, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 19:53:40,578 DEBUG [RS:0;jenkins-hbase4:33367] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@14e72d42, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-03-08 19:53:40,621 DEBUG [RS:0;jenkins-hbase4:33367] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:33367 2023-03-08 19:53:40,626 INFO [RS:0;jenkins-hbase4:33367] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-03-08 19:53:40,626 INFO [RS:0;jenkins-hbase4:33367] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-03-08 19:53:40,626 DEBUG [RS:0;jenkins-hbase4:33367] regionserver.HRegionServer(1023): About to register with Master. 2023-03-08 19:53:40,628 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-03-08 19:53:40,629 INFO [RS:0;jenkins-hbase4:33367] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,39989,1678305217508 with isa=jenkins-hbase4.apache.org/172.31.14.131:33367, startcode=1678305218680 2023-03-08 19:53:40,641 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-03-08 19:53:40,641 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-03-08 19:53:40,642 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-03-08 19:53:40,642 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-03-08 19:53:40,642 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-03-08 19:53:40,642 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-08 19:53:40,642 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-03-08 19:53:40,642 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-08 19:53:40,646 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1678305250646 2023-03-08 19:53:40,648 DEBUG [RS:0;jenkins-hbase4:33367] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-03-08 19:53:40,648 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-03-08 19:53:40,657 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-03-08 19:53:40,658 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-03-08 19:53:40,659 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-03-08 19:53:40,664 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-03-08 19:53:40,668 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-03-08 19:53:40,668 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-03-08 19:53:40,669 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-03-08 19:53:40,669 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-03-08 19:53:40,670 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-03-08 19:53:40,677 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-03-08 19:53:40,678 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-03-08 19:53:40,678 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-03-08 19:53:40,681 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-03-08 19:53:40,681 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-03-08 19:53:40,683 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1678305220683,5,FailOnTimeoutGroup] 2023-03-08 19:53:40,684 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1678305220684,5,FailOnTimeoutGroup] 2023-03-08 19:53:40,684 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-03-08 19:53:40,684 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-03-08 19:53:40,685 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-03-08 19:53:40,685 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-03-08 19:53:40,705 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59665, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-03-08 19:53:40,717 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:40,733 DEBUG [RS:0;jenkins-hbase4:33367] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488 2023-03-08 19:53:40,733 DEBUG [RS:0;jenkins-hbase4:33367] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:42127 2023-03-08 19:53:40,733 DEBUG [RS:0;jenkins-hbase4:33367] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-03-08 19:53:40,739 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-03-08 19:53:40,740 DEBUG [RS:0;jenkins-hbase4:33367] zookeeper.ZKUtil(162): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:40,740 WARN [RS:0;jenkins-hbase4:33367] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-03-08 19:53:40,741 INFO [RS:0;jenkins-hbase4:33367] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-03-08 19:53:40,742 DEBUG [RS:0;jenkins-hbase4:33367] regionserver.HRegionServer(1947): logDir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/WALs/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:40,743 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,33367,1678305218680] 2023-03-08 19:53:40,751 DEBUG [RS:0;jenkins-hbase4:33367] zookeeper.ZKUtil(162): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:40,763 DEBUG [RS:0;jenkins-hbase4:33367] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-03-08 19:53:40,773 INFO [RS:0;jenkins-hbase4:33367] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-03-08 19:53:40,795 INFO [RS:0;jenkins-hbase4:33367] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-03-08 19:53:40,799 INFO [RS:0;jenkins-hbase4:33367] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 19:53:40,799 INFO [RS:0;jenkins-hbase4:33367] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 19:53:40,801 INFO [RS:0;jenkins-hbase4:33367] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-03-08 19:53:40,808 INFO [RS:0;jenkins-hbase4:33367] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=100, unit=MILLISECONDS is enabled. 2023-03-08 19:53:40,809 DEBUG [RS:0;jenkins-hbase4:33367] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-08 19:53:40,809 DEBUG [RS:0;jenkins-hbase4:33367] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-08 19:53:40,809 DEBUG [RS:0;jenkins-hbase4:33367] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-08 19:53:40,809 DEBUG [RS:0;jenkins-hbase4:33367] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-08 19:53:40,809 DEBUG [RS:0;jenkins-hbase4:33367] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-08 19:53:40,809 DEBUG [RS:0;jenkins-hbase4:33367] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-03-08 19:53:40,810 DEBUG [RS:0;jenkins-hbase4:33367] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-08 19:53:40,810 DEBUG [RS:0;jenkins-hbase4:33367] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-08 19:53:40,810 DEBUG [RS:0;jenkins-hbase4:33367] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-08 19:53:40,810 DEBUG [RS:0;jenkins-hbase4:33367] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-08 19:53:40,813 INFO [RS:0;jenkins-hbase4:33367] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-03-08 19:53:40,813 INFO [RS:0;jenkins-hbase4:33367] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-03-08 19:53:40,814 INFO [RS:0;jenkins-hbase4:33367] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-03-08 19:53:40,829 INFO [RS:0;jenkins-hbase4:33367] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-03-08 19:53:40,832 INFO [RS:0;jenkins-hbase4:33367] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33367,1678305218680-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 19:53:40,848 INFO [RS:0;jenkins-hbase4:33367] regionserver.Replication(203): jenkins-hbase4.apache.org,33367,1678305218680 started 2023-03-08 19:53:40,849 INFO [RS:0;jenkins-hbase4:33367] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,33367,1678305218680, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:33367, sessionid=0x100c5b1324f0001 2023-03-08 19:53:40,849 DEBUG [RS:0;jenkins-hbase4:33367] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-03-08 19:53:40,849 DEBUG [RS:0;jenkins-hbase4:33367] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:40,849 DEBUG [RS:0;jenkins-hbase4:33367] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33367,1678305218680' 2023-03-08 19:53:40,849 DEBUG [RS:0;jenkins-hbase4:33367] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:53:40,850 DEBUG [RS:0;jenkins-hbase4:33367] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:53:40,850 DEBUG [RS:0;jenkins-hbase4:33367] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-03-08 19:53:40,850 DEBUG [RS:0;jenkins-hbase4:33367] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-03-08 19:53:40,850 DEBUG [RS:0;jenkins-hbase4:33367] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:40,850 DEBUG [RS:0;jenkins-hbase4:33367] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33367,1678305218680' 2023-03-08 19:53:40,850 DEBUG [RS:0;jenkins-hbase4:33367] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-03-08 19:53:40,851 DEBUG [RS:0;jenkins-hbase4:33367] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-03-08 19:53:40,851 DEBUG [RS:0;jenkins-hbase4:33367] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-03-08 19:53:40,851 INFO [RS:0;jenkins-hbase4:33367] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-03-08 19:53:40,851 INFO [RS:0;jenkins-hbase4:33367] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-03-08 19:53:40,963 INFO [RS:0;jenkins-hbase4:33367] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33367%2C1678305218680, suffix=, logDir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/WALs/jenkins-hbase4.apache.org,33367,1678305218680, archiveDir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/oldWALs, maxLogs=32 2023-03-08 19:53:40,984 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41113,DS-7ccd58a4-abcb-43bf-82b0-b526b26a4701,DISK] 2023-03-08 19:53:40,987 INFO [RS:0;jenkins-hbase4:33367] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/WALs/jenkins-hbase4.apache.org,33367,1678305218680/jenkins-hbase4.apache.org%2C33367%2C1678305218680.1678305220966 2023-03-08 19:53:40,988 DEBUG [RS:0;jenkins-hbase4:33367] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41113,DS-7ccd58a4-abcb-43bf-82b0-b526b26a4701,DISK]] 2023-03-08 19:53:41,101 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-03-08 19:53:41,102 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-03-08 19:53:41,102 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488 2023-03-08 19:53:41,118 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:53:41,120 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-03-08 19:53:41,122 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/hbase/meta/1588230740/info 2023-03-08 19:53:41,123 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-03-08 19:53:41,124 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:53:41,124 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-03-08 19:53:41,127 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/hbase/meta/1588230740/rep_barrier 2023-03-08 19:53:41,127 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-03-08 19:53:41,128 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:53:41,128 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-03-08 19:53:41,130 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/hbase/meta/1588230740/table 2023-03-08 19:53:41,131 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-03-08 19:53:41,132 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:53:41,134 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/hbase/meta/1588230740 2023-03-08 19:53:41,135 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/hbase/meta/1588230740 2023-03-08 19:53:41,140 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-03-08 19:53:41,142 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-03-08 19:53:41,145 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 19:53:41,147 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74360919, jitterRate=0.10806404054164886}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-03-08 19:53:41,147 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-03-08 19:53:41,147 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-03-08 19:53:41,147 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-03-08 19:53:41,147 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-03-08 19:53:41,147 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-03-08 19:53:41,147 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-03-08 19:53:41,148 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-03-08 19:53:41,149 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-03-08 19:53:41,154 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-03-08 19:53:41,154 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-03-08 19:53:41,162 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-03-08 19:53:41,174 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-03-08 19:53:41,177 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-03-08 19:53:41,329 DEBUG [jenkins-hbase4:39989] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-03-08 19:53:41,332 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,33367,1678305218680, state=OPENING 2023-03-08 19:53:41,339 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-03-08 19:53:41,341 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-08 19:53:41,342 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-03-08 19:53:41,344 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,33367,1678305218680}] 2023-03-08 19:53:41,529 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:41,532 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-03-08 19:53:41,535 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59414, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-03-08 19:53:41,547 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-03-08 19:53:41,547 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-03-08 19:53:41,551 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33367%2C1678305218680.meta, suffix=.meta, logDir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/WALs/jenkins-hbase4.apache.org,33367,1678305218680, archiveDir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/oldWALs, maxLogs=32 2023-03-08 19:53:41,567 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41113,DS-7ccd58a4-abcb-43bf-82b0-b526b26a4701,DISK] 2023-03-08 19:53:41,571 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/WALs/jenkins-hbase4.apache.org,33367,1678305218680/jenkins-hbase4.apache.org%2C33367%2C1678305218680.meta.1678305221552.meta 2023-03-08 19:53:41,571 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41113,DS-7ccd58a4-abcb-43bf-82b0-b526b26a4701,DISK]] 2023-03-08 19:53:41,571 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-03-08 19:53:41,573 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-03-08 19:53:41,588 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-03-08 19:53:41,593 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-03-08 19:53:41,599 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-03-08 19:53:41,599 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:53:41,599 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-03-08 19:53:41,599 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-03-08 19:53:41,602 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-03-08 19:53:41,603 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/hbase/meta/1588230740/info 2023-03-08 19:53:41,603 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/hbase/meta/1588230740/info 2023-03-08 19:53:41,604 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-03-08 19:53:41,605 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:53:41,605 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-03-08 19:53:41,607 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/hbase/meta/1588230740/rep_barrier 2023-03-08 19:53:41,607 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/hbase/meta/1588230740/rep_barrier 2023-03-08 19:53:41,607 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-03-08 19:53:41,608 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:53:41,608 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-03-08 19:53:41,610 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/hbase/meta/1588230740/table 2023-03-08 19:53:41,610 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/hbase/meta/1588230740/table 2023-03-08 19:53:41,610 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-03-08 19:53:41,611 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:53:41,613 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/hbase/meta/1588230740 2023-03-08 19:53:41,615 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/hbase/meta/1588230740 2023-03-08 19:53:41,619 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-03-08 19:53:41,622 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-03-08 19:53:41,624 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70119426, jitterRate=0.04486086964607239}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-03-08 19:53:41,625 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-03-08 19:53:41,635 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1678305221520 2023-03-08 19:53:41,656 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-03-08 19:53:41,657 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-03-08 19:53:41,658 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,33367,1678305218680, state=OPEN 2023-03-08 19:53:41,661 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-03-08 19:53:41,661 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-03-08 19:53:41,666 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-03-08 19:53:41,666 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,33367,1678305218680 in 317 msec 2023-03-08 19:53:41,672 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-03-08 19:53:41,672 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 505 msec 2023-03-08 19:53:41,678 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 1.1360 sec 2023-03-08 19:53:41,678 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1678305221678, completionTime=-1 2023-03-08 19:53:41,678 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-03-08 19:53:41,679 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-03-08 19:53:41,744 DEBUG [hconnection-0x1ac1bbe3-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 19:53:41,748 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59430, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 19:53:41,767 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=1 2023-03-08 19:53:41,767 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1678305281767 2023-03-08 19:53:41,767 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1678305341767 2023-03-08 19:53:41,767 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 88 msec 2023-03-08 19:53:41,792 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39989,1678305217508-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 19:53:41,792 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39989,1678305217508-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-03-08 19:53:41,792 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39989,1678305217508-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-03-08 19:53:41,794 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:39989, period=300000, unit=MILLISECONDS is enabled. 2023-03-08 19:53:41,795 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-03-08 19:53:41,802 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-03-08 19:53:41,811 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-03-08 19:53:41,812 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-03-08 19:53:41,824 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-03-08 19:53:41,826 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 19:53:41,830 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 19:53:41,853 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/hbase/namespace/3e027aaacb7795251255fdd7284ec165 2023-03-08 19:53:41,856 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/hbase/namespace/3e027aaacb7795251255fdd7284ec165 empty. 2023-03-08 19:53:41,857 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/hbase/namespace/3e027aaacb7795251255fdd7284ec165 2023-03-08 19:53:41,857 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-03-08 19:53:42,281 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-03-08 19:53:42,283 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 3e027aaacb7795251255fdd7284ec165, NAME => 'hbase:namespace,,1678305221812.3e027aaacb7795251255fdd7284ec165.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp 2023-03-08 19:53:42,695 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1678305221812.3e027aaacb7795251255fdd7284ec165.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:53:42,696 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 3e027aaacb7795251255fdd7284ec165, disabling compactions & flushes 2023-03-08 19:53:42,696 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1678305221812.3e027aaacb7795251255fdd7284ec165. 2023-03-08 19:53:42,696 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1678305221812.3e027aaacb7795251255fdd7284ec165. 2023-03-08 19:53:42,696 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1678305221812.3e027aaacb7795251255fdd7284ec165. after waiting 0 ms 2023-03-08 19:53:42,696 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1678305221812.3e027aaacb7795251255fdd7284ec165. 2023-03-08 19:53:42,696 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1678305221812.3e027aaacb7795251255fdd7284ec165. 2023-03-08 19:53:42,696 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 3e027aaacb7795251255fdd7284ec165: 2023-03-08 19:53:42,701 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 19:53:42,717 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1678305221812.3e027aaacb7795251255fdd7284ec165.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1678305222705"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678305222705"}]},"ts":"1678305222705"} 2023-03-08 19:53:42,745 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-08 19:53:42,747 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 19:53:42,752 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678305222747"}]},"ts":"1678305222747"} 2023-03-08 19:53:42,756 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-03-08 19:53:42,766 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=3e027aaacb7795251255fdd7284ec165, ASSIGN}] 2023-03-08 19:53:42,769 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=3e027aaacb7795251255fdd7284ec165, ASSIGN 2023-03-08 19:53:42,771 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=3e027aaacb7795251255fdd7284ec165, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33367,1678305218680; forceNewPlan=false, retain=false 2023-03-08 19:53:42,922 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=3e027aaacb7795251255fdd7284ec165, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:42,922 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1678305221812.3e027aaacb7795251255fdd7284ec165.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1678305222921"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678305222921"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678305222921"}]},"ts":"1678305222921"} 2023-03-08 19:53:42,926 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 3e027aaacb7795251255fdd7284ec165, server=jenkins-hbase4.apache.org,33367,1678305218680}] 2023-03-08 19:53:43,087 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1678305221812.3e027aaacb7795251255fdd7284ec165. 2023-03-08 19:53:43,088 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 3e027aaacb7795251255fdd7284ec165, NAME => 'hbase:namespace,,1678305221812.3e027aaacb7795251255fdd7284ec165.', STARTKEY => '', ENDKEY => ''} 2023-03-08 19:53:43,089 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 3e027aaacb7795251255fdd7284ec165 2023-03-08 19:53:43,089 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1678305221812.3e027aaacb7795251255fdd7284ec165.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:53:43,089 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 3e027aaacb7795251255fdd7284ec165 2023-03-08 19:53:43,089 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 3e027aaacb7795251255fdd7284ec165 2023-03-08 19:53:43,091 INFO [StoreOpener-3e027aaacb7795251255fdd7284ec165-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 3e027aaacb7795251255fdd7284ec165 2023-03-08 19:53:43,093 DEBUG [StoreOpener-3e027aaacb7795251255fdd7284ec165-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/hbase/namespace/3e027aaacb7795251255fdd7284ec165/info 2023-03-08 19:53:43,093 DEBUG [StoreOpener-3e027aaacb7795251255fdd7284ec165-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/hbase/namespace/3e027aaacb7795251255fdd7284ec165/info 2023-03-08 19:53:43,094 INFO [StoreOpener-3e027aaacb7795251255fdd7284ec165-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3e027aaacb7795251255fdd7284ec165 columnFamilyName info 2023-03-08 19:53:43,094 INFO [StoreOpener-3e027aaacb7795251255fdd7284ec165-1] regionserver.HStore(310): Store=3e027aaacb7795251255fdd7284ec165/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:53:43,096 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/hbase/namespace/3e027aaacb7795251255fdd7284ec165 2023-03-08 19:53:43,097 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/hbase/namespace/3e027aaacb7795251255fdd7284ec165 2023-03-08 19:53:43,102 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 3e027aaacb7795251255fdd7284ec165 2023-03-08 19:53:43,105 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/hbase/namespace/3e027aaacb7795251255fdd7284ec165/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 19:53:43,106 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 3e027aaacb7795251255fdd7284ec165; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62577927, jitterRate=-0.06751622259616852}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 19:53:43,106 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 3e027aaacb7795251255fdd7284ec165: 2023-03-08 19:53:43,108 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1678305221812.3e027aaacb7795251255fdd7284ec165., pid=6, masterSystemTime=1678305223080 2023-03-08 19:53:43,112 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1678305221812.3e027aaacb7795251255fdd7284ec165. 2023-03-08 19:53:43,112 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1678305221812.3e027aaacb7795251255fdd7284ec165. 2023-03-08 19:53:43,114 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=3e027aaacb7795251255fdd7284ec165, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:43,114 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1678305221812.3e027aaacb7795251255fdd7284ec165.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1678305223113"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678305223113"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678305223113"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678305223113"}]},"ts":"1678305223113"} 2023-03-08 19:53:43,121 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-03-08 19:53:43,121 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 3e027aaacb7795251255fdd7284ec165, server=jenkins-hbase4.apache.org,33367,1678305218680 in 192 msec 2023-03-08 19:53:43,124 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-03-08 19:53:43,125 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=3e027aaacb7795251255fdd7284ec165, ASSIGN in 355 msec 2023-03-08 19:53:43,126 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 19:53:43,126 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678305223126"}]},"ts":"1678305223126"} 2023-03-08 19:53:43,129 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-03-08 19:53:43,132 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 19:53:43,134 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 1.3190 sec 2023-03-08 19:53:43,228 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-03-08 19:53:43,231 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-03-08 19:53:43,231 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-08 19:53:43,282 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-03-08 19:53:43,304 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-03-08 19:53:43,311 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 39 msec 2023-03-08 19:53:43,318 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-03-08 19:53:43,328 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-03-08 19:53:43,333 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 14 msec 2023-03-08 19:53:43,344 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-03-08 19:53:43,347 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-03-08 19:53:43,347 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 4.587sec 2023-03-08 19:53:43,349 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-03-08 19:53:43,351 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-03-08 19:53:43,351 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-03-08 19:53:43,352 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39989,1678305217508-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-03-08 19:53:43,353 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39989,1678305217508-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-03-08 19:53:43,364 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-03-08 19:53:43,434 DEBUG [Listener at localhost/34049] zookeeper.ReadOnlyZKClient(139): Connect 0x6decc8b7 to 127.0.0.1:56751 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 19:53:43,439 DEBUG [Listener at localhost/34049] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@515b7931, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 19:53:43,454 DEBUG [hconnection-0x1b386a6b-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 19:53:43,469 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43930, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 19:53:43,479 INFO [Listener at localhost/34049] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,39989,1678305217508 2023-03-08 19:53:43,479 DEBUG [Listener at localhost/34049] zookeeper.ReadOnlyZKClient(139): Connect 0x06140021 to 127.0.0.1:56751 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 19:53:43,486 DEBUG [Listener at localhost/34049] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4e2f2d5d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 19:53:43,486 INFO [Listener at localhost/34049] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-08 19:53:43,509 INFO [Listener at localhost/34049] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=211, OpenFileDescriptor=475, MaxFileDescriptor=60000, SystemLoadAverage=330, ProcessCount=170, AvailableMemoryMB=8590 2023-03-08 19:53:43,520 DEBUG [Listener at localhost/34049] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 19:53:43,523 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53838, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 19:53:43,536 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-08 19:53:43,538 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:43,540 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 19:53:43,543 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 19:53:43,545 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthly" procId is: 9 2023-03-08 19:53:43,546 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436 2023-03-08 19:53:43,547 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436 empty. 2023-03-08 19:53:43,549 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436 2023-03-08 19:53:43,549 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthly regions 2023-03-08 19:53:43,555 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-08 19:53:43,566 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-03-08 19:53:43,568 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(7675): creating {ENCODED => 4ad1e88a375de73ef8abab9078c9a436, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp 2023-03-08 19:53:43,660 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-08 19:53:43,863 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-08 19:53:44,009 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:53:44,010 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1604): Closing 4ad1e88a375de73ef8abab9078c9a436, disabling compactions & flushes 2023-03-08 19:53:44,010 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:53:44,010 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:53:44,010 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. after waiting 0 ms 2023-03-08 19:53:44,010 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:53:44,010 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:53:44,010 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1558): Region close journal for 4ad1e88a375de73ef8abab9078c9a436: 2023-03-08 19:53:44,014 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 19:53:44,016 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678305224016"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678305224016"}]},"ts":"1678305224016"} 2023-03-08 19:53:44,019 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-08 19:53:44,020 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 19:53:44,020 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678305224020"}]},"ts":"1678305224020"} 2023-03-08 19:53:44,023 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLING in hbase:meta 2023-03-08 19:53:44,027 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=4ad1e88a375de73ef8abab9078c9a436, ASSIGN}] 2023-03-08 19:53:44,029 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=4ad1e88a375de73ef8abab9078c9a436, ASSIGN 2023-03-08 19:53:44,030 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=4ad1e88a375de73ef8abab9078c9a436, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33367,1678305218680; forceNewPlan=false, retain=false 2023-03-08 19:53:44,165 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-08 19:53:44,181 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=4ad1e88a375de73ef8abab9078c9a436, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:44,182 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678305224181"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678305224181"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678305224181"}]},"ts":"1678305224181"} 2023-03-08 19:53:44,185 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 4ad1e88a375de73ef8abab9078c9a436, server=jenkins-hbase4.apache.org,33367,1678305218680}] 2023-03-08 19:53:44,344 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:53:44,344 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 4ad1e88a375de73ef8abab9078c9a436, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436.', STARTKEY => '', ENDKEY => ''} 2023-03-08 19:53:44,344 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 4ad1e88a375de73ef8abab9078c9a436 2023-03-08 19:53:44,344 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:53:44,344 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 4ad1e88a375de73ef8abab9078c9a436 2023-03-08 19:53:44,344 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 4ad1e88a375de73ef8abab9078c9a436 2023-03-08 19:53:44,347 INFO [StoreOpener-4ad1e88a375de73ef8abab9078c9a436-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 4ad1e88a375de73ef8abab9078c9a436 2023-03-08 19:53:44,350 DEBUG [StoreOpener-4ad1e88a375de73ef8abab9078c9a436-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1 2023-03-08 19:53:44,350 DEBUG [StoreOpener-4ad1e88a375de73ef8abab9078c9a436-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1 2023-03-08 19:53:44,353 INFO [StoreOpener-4ad1e88a375de73ef8abab9078c9a436-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4ad1e88a375de73ef8abab9078c9a436 columnFamilyName family1 2023-03-08 19:53:44,355 INFO [StoreOpener-4ad1e88a375de73ef8abab9078c9a436-1] regionserver.HStore(310): Store=4ad1e88a375de73ef8abab9078c9a436/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:53:44,358 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436 2023-03-08 19:53:44,359 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436 2023-03-08 19:53:44,363 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 4ad1e88a375de73ef8abab9078c9a436 2023-03-08 19:53:44,367 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 19:53:44,368 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 4ad1e88a375de73ef8abab9078c9a436; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59253502, jitterRate=-0.11705401539802551}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 19:53:44,368 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 4ad1e88a375de73ef8abab9078c9a436: 2023-03-08 19:53:44,369 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436., pid=11, masterSystemTime=1678305224338 2023-03-08 19:53:44,372 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:53:44,372 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:53:44,373 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=4ad1e88a375de73ef8abab9078c9a436, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:44,374 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678305224373"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678305224373"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678305224373"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678305224373"}]},"ts":"1678305224373"} 2023-03-08 19:53:44,380 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-03-08 19:53:44,380 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 4ad1e88a375de73ef8abab9078c9a436, server=jenkins-hbase4.apache.org,33367,1678305218680 in 191 msec 2023-03-08 19:53:44,383 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-03-08 19:53:44,383 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=4ad1e88a375de73ef8abab9078c9a436, ASSIGN in 353 msec 2023-03-08 19:53:44,384 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 19:53:44,384 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678305224384"}]},"ts":"1678305224384"} 2023-03-08 19:53:44,386 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLED in hbase:meta 2023-03-08 19:53:44,389 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 19:53:44,391 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 853 msec 2023-03-08 19:53:44,667 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-08 19:53:44,668 INFO [Listener at localhost/34049] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthly, procId: 9 completed 2023-03-08 19:53:44,685 DEBUG [hconnection-0x3951a133-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 19:53:44,688 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43938, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 19:53:44,831 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:53:44,838 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:44,839 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 19:53:44,839 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:53:44,839 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-08 19:53:44,839 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-08 19:53:44,840 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:44,840 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:44,843 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:53:44,843 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:44,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:53:44,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:53:44,843 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:44,844 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:53:44,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:44,844 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:44,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:53:44,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:44,845 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:44,847 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:44,847 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-08 19:53:44,847 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:53:44,847 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-08 19:53:44,848 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:53:44,848 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:53:44,848 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:53:44,849 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. started... 2023-03-08 19:53:44,850 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 4ad1e88a375de73ef8abab9078c9a436 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-08 19:53:45,384 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511300f5b82cfb5ce41389fa3db34e33c098d to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511300f5b82cfb5ce41389fa3db34e33c098d 2023-03-08 19:53:45,398 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/.tmp/family1/0435e74f700e4bf3a70e98f4689f8070 2023-03-08 19:53:45,408 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/.tmp/family1/0435e74f700e4bf3a70e98f4689f8070 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/0435e74f700e4bf3a70e98f4689f8070 2023-03-08 19:53:45,417 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/0435e74f700e4bf3a70e98f4689f8070, entries=1000, sequenceid=5, filesize=175.8 K 2023-03-08 19:53:45,419 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 4ad1e88a375de73ef8abab9078c9a436 in 569ms, sequenceid=5, compaction requested=false 2023-03-08 19:53:45,421 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 19:53:45,423 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4ad1e88a375de73ef8abab9078c9a436: 2023-03-08 19:53:45,424 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:53:45,424 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:53:45,424 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:53:45,424 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:45,424 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-08 19:53:45,424 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-08 19:53:45,427 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:45,427 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:45,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:45,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:53:45,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:53:45,427 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:45,427 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:53:45,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:53:45,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:53:45,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:45,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:45,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:53:45,429 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-08 19:53:45,429 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7c6e00a8[Count = 0] remaining members to acquire global barrier 2023-03-08 19:53:45,429 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-08 19:53:45,430 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:45,432 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:45,433 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:45,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:45,433 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-08 19:53:45,433 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-08 19:53:45,433 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:53:45,433 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:45,433 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:53:45,436 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-08 19:53:45,436 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:45,436 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:53:45,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:45,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:53:45,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:53:45,436 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-08 19:53:45,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:53:45,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:53:45,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:45,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:45,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:53:45,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:45,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:45,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:53:45,440 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-08 19:53:45,440 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-08 19:53:45,440 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:53:45,440 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:53:45,440 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:45,440 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:53:45,443 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:45,443 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:45,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:45,443 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:53:45,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:45,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:45,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:53:45,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:53:45,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:53:45,444 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:45,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:53:45,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:53:45,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:45,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:45,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:53:45,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:45,446 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:45,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:45,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:53:45,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:45,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:45,454 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:45,454 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:53:45,454 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:45,454 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:53:45,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:53:45,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:53:45,454 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:53:45,454 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:45,454 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 19:53:45,455 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:45,455 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:53:45,455 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:45,455 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:45,455 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:45,455 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:53:45,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:53:45,457 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-08 19:53:45,457 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:53:45,557 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:53:45,562 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:53:45,572 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:53:45,575 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-08 19:53:45,575 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:45,575 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 19:53:45,575 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:53:45,575 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-08 19:53:45,575 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-08 19:53:45,576 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:45,576 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:45,578 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:53:45,578 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:45,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:53:45,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:53:45,578 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:45,578 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:53:45,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:45,579 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:45,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:53:45,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:45,579 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:45,580 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-08 19:53:45,580 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:45,580 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-08 19:53:45,580 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:53:45,580 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-08 19:53:45,580 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:53:45,580 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:53:45,581 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:53:45,581 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. started... 2023-03-08 19:53:45,581 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 4ad1e88a375de73ef8abab9078c9a436 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-08 19:53:46,007 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112885b1d054e50f4fbab508f15b87ee94db to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015112885b1d054e50f4fbab508f15b87ee94db 2023-03-08 19:53:46,420 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/.tmp/family1/d7bc8921991b4b9e869f5f7ec27724b0 2023-03-08 19:53:46,429 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/.tmp/family1/d7bc8921991b4b9e869f5f7ec27724b0 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/d7bc8921991b4b9e869f5f7ec27724b0 2023-03-08 19:53:46,438 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/d7bc8921991b4b9e869f5f7ec27724b0, entries=1, sequenceid=9, filesize=5.0 K 2023-03-08 19:53:46,439 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 4ad1e88a375de73ef8abab9078c9a436 in 858ms, sequenceid=9, compaction requested=false 2023-03-08 19:53:46,439 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4ad1e88a375de73ef8abab9078c9a436: 2023-03-08 19:53:46,440 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:53:46,440 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:53:46,440 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:53:46,440 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:46,440 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-08 19:53:46,440 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-08 19:53:46,443 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:46,443 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:46,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:46,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:53:46,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:53:46,444 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:46,444 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:53:46,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:53:46,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:53:46,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:46,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:46,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:53:46,446 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-08 19:53:46,446 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@55f76c11[Count = 0] remaining members to acquire global barrier 2023-03-08 19:53:46,446 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-08 19:53:46,446 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:46,448 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:46,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:46,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:46,448 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-08 19:53:46,448 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-08 19:53:46,448 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:53:46,448 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:46,448 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:53:46,450 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-08 19:53:46,450 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:46,450 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:53:46,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:46,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:53:46,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:53:46,451 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-08 19:53:46,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:53:46,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:53:46,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:46,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:46,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:53:46,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:46,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:46,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:53:46,455 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-08 19:53:46,455 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-08 19:53:46,455 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:53:46,455 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:53:46,455 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:46,455 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:53:46,462 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:46,462 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:46,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:46,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:46,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:53:46,462 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:53:46,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:53:46,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:46,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:53:46,463 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:46,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:53:46,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:53:46,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:46,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:46,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:53:46,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:46,465 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:46,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:46,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:53:46,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:46,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:46,470 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:53:46,470 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:46,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:53:46,470 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:53:46,470 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-08 19:53:46,470 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:46,470 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:53:46,470 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:46,470 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 19:53:46,471 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:53:46,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:53:46,471 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:46,471 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:46,471 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:46,471 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:46,471 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-08 19:53:46,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:53:46,471 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:53:46,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:53:46,571 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:53:46,573 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:53:46,585 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:53:46,589 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-08 19:53:46,589 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:46,589 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 19:53:46,589 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:53:46,590 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-08 19:53:46,590 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-08 19:53:46,590 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:46,591 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:46,593 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:53:46,593 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:46,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:53:46,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:53:46,594 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:46,594 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:53:46,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:46,595 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:46,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:53:46,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:46,595 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:46,596 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-08 19:53:46,596 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:46,596 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-08 19:53:46,596 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:53:46,596 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-08 19:53:46,597 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:53:46,597 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:53:46,597 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:53:46,597 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. started... 2023-03-08 19:53:46,597 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 4ad1e88a375de73ef8abab9078c9a436 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:53:46,651 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201faa8f7bf04cd4d08bc3dd154ed269526 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201faa8f7bf04cd4d08bc3dd154ed269526 2023-03-08 19:53:47,065 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/.tmp/family1/27431a5ef048444da16f258fd0926546 2023-03-08 19:53:47,076 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/.tmp/family1/27431a5ef048444da16f258fd0926546 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/27431a5ef048444da16f258fd0926546 2023-03-08 19:53:47,089 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/27431a5ef048444da16f258fd0926546, entries=1, sequenceid=13, filesize=5.0 K 2023-03-08 19:53:47,090 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 4ad1e88a375de73ef8abab9078c9a436 in 493ms, sequenceid=13, compaction requested=true 2023-03-08 19:53:47,090 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4ad1e88a375de73ef8abab9078c9a436: 2023-03-08 19:53:47,091 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:53:47,091 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:53:47,091 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:53:47,091 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:47,091 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-08 19:53:47,091 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-08 19:53:47,094 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:47,094 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:47,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:47,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:53:47,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:53:47,095 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:47,095 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:53:47,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:53:47,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:53:47,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:47,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:47,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:53:47,097 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-08 19:53:47,097 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@455683aa[Count = 0] remaining members to acquire global barrier 2023-03-08 19:53:47,097 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-08 19:53:47,098 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:47,100 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:47,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:47,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:47,100 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-08 19:53:47,100 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-08 19:53:47,100 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:53:47,100 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:47,100 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:53:47,103 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-08 19:53:47,103 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:47,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:47,103 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:53:47,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:53:47,103 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-08 19:53:47,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:53:47,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:53:47,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:53:47,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:47,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:47,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:53:47,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:47,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:47,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:53:47,108 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-08 19:53:47,108 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-08 19:53:47,108 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:53:47,108 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:53:47,108 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:47,108 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:53:47,110 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:47,110 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:47,110 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:53:47,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:47,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:53:47,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:53:47,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:53:47,112 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:47,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:47,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:53:47,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:47,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:47,114 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:47,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:53:47,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:47,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:47,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:47,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:47,119 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:47,119 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:53:47,119 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:47,119 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-08 19:53:47,119 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:47,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:53:47,119 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:47,119 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:53:47,119 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:47,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:53:47,120 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:47,119 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 19:53:47,120 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:53:47,120 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:47,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:53:47,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:53:47,121 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-08 19:53:47,121 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:53:47,135 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 19:53:47,196 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-03-08 19:53:47,197 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-03-08 19:53:47,221 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:53:47,222 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:53:47,230 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:53:47,232 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-08 19:53:47,232 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:47,232 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 19:53:47,232 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:53:47,233 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-08 19:53:47,233 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-08 19:53:47,234 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:47,234 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:47,236 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:53:47,236 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:47,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:53:47,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:53:47,237 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:47,237 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:53:47,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:47,237 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:47,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:53:47,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:47,238 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:47,238 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-08 19:53:47,238 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:47,238 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-08 19:53:47,238 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:53:47,239 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-08 19:53:47,239 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:53:47,239 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:53:47,239 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:53:47,239 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. started... 2023-03-08 19:53:47,239 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 4ad1e88a375de73ef8abab9078c9a436 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:53:47,660 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-03-08 19:53:47,660 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-03-08 19:53:47,664 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512055640501452aa45cbaaf465648d56e3ad to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512055640501452aa45cbaaf465648d56e3ad 2023-03-08 19:53:48,077 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/.tmp/family1/865e3957809340c1904c484f5006e36b 2023-03-08 19:53:48,086 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/.tmp/family1/865e3957809340c1904c484f5006e36b as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/865e3957809340c1904c484f5006e36b 2023-03-08 19:53:48,095 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/865e3957809340c1904c484f5006e36b, entries=1, sequenceid=17, filesize=5.0 K 2023-03-08 19:53:48,097 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 4ad1e88a375de73ef8abab9078c9a436 in 858ms, sequenceid=17, compaction requested=true 2023-03-08 19:53:48,097 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4ad1e88a375de73ef8abab9078c9a436: 2023-03-08 19:53:48,097 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:53:48,097 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:53:48,098 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:53:48,098 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:48,098 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-08 19:53:48,098 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-08 19:53:48,101 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:48,101 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:48,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:48,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:53:48,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:53:48,101 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:48,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:53:48,101 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:53:48,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:53:48,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:48,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:48,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:53:48,104 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-08 19:53:48,104 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3716b809[Count = 0] remaining members to acquire global barrier 2023-03-08 19:53:48,104 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-08 19:53:48,104 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:48,105 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:48,105 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:48,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:48,105 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-08 19:53:48,105 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-08 19:53:48,106 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:53:48,106 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:48,106 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:53:48,108 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-08 19:53:48,108 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:53:48,108 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-08 19:53:48,108 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:48,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:48,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:53:48,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:53:48,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:53:48,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:53:48,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:48,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:48,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:53:48,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:48,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:48,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:53:48,113 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-08 19:53:48,113 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-08 19:53:48,113 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:53:48,113 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:53:48,113 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:48,113 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:53:48,116 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:48,116 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:48,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:48,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:48,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:53:48,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:53:48,116 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:53:48,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:48,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:53:48,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:53:48,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:53:48,117 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:48,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:48,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:48,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:53:48,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:48,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:48,119 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:48,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:53:48,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:48,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:48,124 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:48,124 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:53:48,124 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:48,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:53:48,124 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 19:53:48,124 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:53:48,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:53:48,124 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:53:48,124 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:48,124 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:53:48,125 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:48,125 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:48,125 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:48,125 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:48,125 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-08 19:53:48,126 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:53:48,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:53:48,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:53:48,226 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:53:48,228 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:53:48,236 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:53:48,238 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-08 19:53:48,239 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:48,239 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 19:53:48,239 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:53:48,239 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-08 19:53:48,239 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-08 19:53:48,240 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:48,240 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:48,241 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:48,241 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:53:48,241 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:53:48,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:53:48,242 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:48,242 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:53:48,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:48,242 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:48,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:53:48,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:48,243 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:48,243 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-08 19:53:48,243 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:48,243 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-08 19:53:48,243 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:53:48,244 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-08 19:53:48,244 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:53:48,244 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:53:48,244 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:53:48,244 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. started... 2023-03-08 19:53:48,244 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 4ad1e88a375de73ef8abab9078c9a436 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:53:48,667 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512284c660201c5ac4c859aa1e35c2f397165 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512284c660201c5ac4c859aa1e35c2f397165 2023-03-08 19:53:49,081 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/.tmp/family1/6c4321ddc714451f82661dfe2aad1aac 2023-03-08 19:53:49,090 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/.tmp/family1/6c4321ddc714451f82661dfe2aad1aac as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/6c4321ddc714451f82661dfe2aad1aac 2023-03-08 19:53:49,099 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/6c4321ddc714451f82661dfe2aad1aac, entries=1, sequenceid=21, filesize=5.0 K 2023-03-08 19:53:49,101 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 4ad1e88a375de73ef8abab9078c9a436 in 857ms, sequenceid=21, compaction requested=true 2023-03-08 19:53:49,101 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4ad1e88a375de73ef8abab9078c9a436: 2023-03-08 19:53:49,101 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:53:49,101 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:53:49,102 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:53:49,102 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:49,102 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-08 19:53:49,102 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-08 19:53:49,108 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:49,108 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:49,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:49,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:53:49,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:53:49,109 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:49,109 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:53:49,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:53:49,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:53:49,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:49,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:49,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:53:49,111 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-08 19:53:49,111 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3bf18503[Count = 0] remaining members to acquire global barrier 2023-03-08 19:53:49,111 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-08 19:53:49,111 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:49,113 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:49,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:49,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:49,114 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-08 19:53:49,114 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-08 19:53:49,114 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:49,114 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:53:49,114 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:53:49,116 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:49,116 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-08 19:53:49,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:49,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:53:49,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:53:49,116 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:53:49,116 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-08 19:53:49,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:53:49,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:53:49,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:49,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:49,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:53:49,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:49,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:49,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:53:49,120 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-08 19:53:49,120 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-08 19:53:49,120 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:53:49,120 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:53:49,120 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:49,120 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:53:49,124 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:49,124 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:49,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:49,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:49,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:53:49,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:53:49,124 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:53:49,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:49,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:53:49,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:53:49,125 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:49,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:53:49,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:49,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:49,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:53:49,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:49,126 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:49,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:49,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:53:49,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:49,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:49,131 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:49,131 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:49,131 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:53:49,131 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:49,131 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:53:49,131 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:53:49,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:53:49,131 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 19:53:49,131 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:53:49,131 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:49,131 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:53:49,131 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:49,131 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:49,131 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:53:49,131 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-08 19:53:49,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:53:49,132 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-08 19:53:49,131 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:49,132 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:53:49,232 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:53:49,234 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:53:49,241 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:53:49,244 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-08 19:53:49,244 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:49,244 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 19:53:49,244 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:53:49,245 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-08 19:53:49,245 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-08 19:53:49,245 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:49,245 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:49,247 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:49,247 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:53:49,247 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:53:49,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:53:49,247 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:49,247 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:53:49,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:49,248 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:49,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:53:49,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:49,248 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:49,249 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-08 19:53:49,249 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:49,249 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-08 19:53:49,249 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:53:49,249 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-08 19:53:49,250 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:53:49,250 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:53:49,250 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:53:49,250 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. started... 2023-03-08 19:53:49,251 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 4ad1e88a375de73ef8abab9078c9a436 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:53:49,688 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512316ddba1fa3b0a4f6da5c6261316cf19f4 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512316ddba1fa3b0a4f6da5c6261316cf19f4 2023-03-08 19:53:50,105 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/.tmp/family1/cb9806e2e04b4d8e85ddbd1bc8a679ff 2023-03-08 19:53:50,125 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/.tmp/family1/cb9806e2e04b4d8e85ddbd1bc8a679ff as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/cb9806e2e04b4d8e85ddbd1bc8a679ff 2023-03-08 19:53:50,136 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/cb9806e2e04b4d8e85ddbd1bc8a679ff, entries=1, sequenceid=25, filesize=5.0 K 2023-03-08 19:53:50,137 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 4ad1e88a375de73ef8abab9078c9a436 in 886ms, sequenceid=25, compaction requested=true 2023-03-08 19:53:50,138 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4ad1e88a375de73ef8abab9078c9a436: 2023-03-08 19:53:50,138 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:53:50,138 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:53:50,138 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:53:50,138 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,138 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-08 19:53:50,138 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-08 19:53:50,141 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:53:50,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:53:50,141 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:53:50,142 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,142 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:53:50,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:53:50,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:53:50,143 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-08 19:53:50,144 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5508f188[Count = 0] remaining members to acquire global barrier 2023-03-08 19:53:50,144 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-08 19:53:50,144 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,145 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,145 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-08 19:53:50,145 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-08 19:53:50,145 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:53:50,145 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,146 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:53:50,148 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,148 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-08 19:53:50,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:53:50,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:53:50,148 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:53:50,149 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-08 19:53:50,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:53:50,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:53:50,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:53:50,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:53:50,154 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-08 19:53:50,154 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-08 19:53:50,154 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:53:50,154 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:53:50,154 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,154 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:53:50,156 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,156 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:53:50,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,156 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:53:50,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:53:50,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:53:50,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:53:50,157 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:53:50,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:53:50,160 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:53:50,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,164 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:53:50,164 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,164 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:53:50,164 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:53:50,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:53:50,164 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 19:53:50,164 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:53:50,164 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:53:50,164 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,165 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,165 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,165 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,165 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,165 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-08 19:53:50,165 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:53:50,165 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-08 19:53:50,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:53:50,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:53:50,266 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:53:50,267 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:53:50,277 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:53:50,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-08 19:53:50,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,280 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 19:53:50,280 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:53:50,281 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-08 19:53:50,281 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-08 19:53:50,282 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,282 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,283 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:53:50,283 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,283 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:53:50,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:53:50,284 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,284 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:53:50,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,284 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:53:50,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,285 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,285 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-08 19:53:50,285 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,285 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-08 19:53:50,285 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:53:50,286 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-08 19:53:50,286 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:53:50,286 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:53:50,286 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:53:50,286 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. started... 2023-03-08 19:53:50,286 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 4ad1e88a375de73ef8abab9078c9a436 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:53:50,308 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601016d1cf375f3664352be78a22a9de8b418 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601016d1cf375f3664352be78a22a9de8b418 2023-03-08 19:53:50,718 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/.tmp/family1/75a5720011f34e3da146711c8486e003 2023-03-08 19:53:50,727 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/.tmp/family1/75a5720011f34e3da146711c8486e003 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/75a5720011f34e3da146711c8486e003 2023-03-08 19:53:50,736 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/75a5720011f34e3da146711c8486e003, entries=1, sequenceid=29, filesize=5.0 K 2023-03-08 19:53:50,737 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 4ad1e88a375de73ef8abab9078c9a436 in 451ms, sequenceid=29, compaction requested=true 2023-03-08 19:53:50,738 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4ad1e88a375de73ef8abab9078c9a436: 2023-03-08 19:53:50,738 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:53:50,738 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:53:50,739 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:53:50,739 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,739 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-08 19:53:50,739 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-08 19:53:50,743 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,743 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:53:50,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:53:50,744 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,744 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:53:50,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:53:50,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:53:50,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:53:50,746 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-08 19:53:50,746 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@24b26350[Count = 0] remaining members to acquire global barrier 2023-03-08 19:53:50,746 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-08 19:53:50,746 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,747 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,747 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-08 19:53:50,747 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-08 19:53:50,747 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,748 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:53:50,747 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:53:50,750 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-08 19:53:50,750 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,750 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:53:50,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:53:50,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:53:50,750 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-08 19:53:50,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:53:50,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:53:50,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:53:50,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:53:50,755 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-08 19:53:50,755 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-08 19:53:50,755 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:53:50,755 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:53:50,755 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,755 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:53:50,757 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,757 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:53:50,757 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:53:50,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:53:50,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:53:50,758 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:53:50,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:53:50,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:53:50,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,761 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:53:50,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,768 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:53:50,768 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:53:50,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:53:50,768 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-08 19:53:50,768 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,768 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 19:53:50,769 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:53:50,768 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:53:50,768 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:53:50,769 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,769 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,769 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,769 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,769 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,769 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-08 19:53:50,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:53:50,770 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:53:50,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:53:50,870 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:53:50,871 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:53:50,878 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:53:50,881 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-08 19:53:50,881 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,881 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 19:53:50,881 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:53:50,881 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-08 19:53:50,881 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-08 19:53:50,882 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,882 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,884 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:53:50,884 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:53:50,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:53:50,884 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,884 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:53:50,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,885 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:53:50,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,885 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,885 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-08 19:53:50,885 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,885 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-08 19:53:50,885 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:53:50,886 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-08 19:53:50,886 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:53:50,886 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:53:50,886 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:53:50,886 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. started... 2023-03-08 19:53:50,886 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 4ad1e88a375de73ef8abab9078c9a436 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:53:50,905 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010367ee4bebfce54c8ebd6ad68637323bda to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010367ee4bebfce54c8ebd6ad68637323bda 2023-03-08 19:53:50,912 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/.tmp/family1/73f8831bdc3e472c8dd43a897d3f0d03 2023-03-08 19:53:50,919 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/.tmp/family1/73f8831bdc3e472c8dd43a897d3f0d03 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/73f8831bdc3e472c8dd43a897d3f0d03 2023-03-08 19:53:50,926 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/73f8831bdc3e472c8dd43a897d3f0d03, entries=1, sequenceid=33, filesize=5.0 K 2023-03-08 19:53:50,927 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 4ad1e88a375de73ef8abab9078c9a436 in 41ms, sequenceid=33, compaction requested=true 2023-03-08 19:53:50,927 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4ad1e88a375de73ef8abab9078c9a436: 2023-03-08 19:53:50,927 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:53:50,927 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:53:50,928 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:53:50,928 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,928 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-08 19:53:50,928 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-08 19:53:50,930 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,930 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:53:50,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:53:50,930 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,930 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:53:50,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:53:50,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:53:50,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:53:50,932 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-08 19:53:50,932 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2666513f[Count = 0] remaining members to acquire global barrier 2023-03-08 19:53:50,932 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-08 19:53:50,932 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,934 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,935 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-08 19:53:50,935 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-08 19:53:50,935 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:53:50,935 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,935 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:53:50,936 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,936 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-08 19:53:50,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:53:50,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:53:50,936 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:53:50,936 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-08 19:53:50,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:53:50,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:53:50,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:53:50,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:53:50,939 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-08 19:53:50,939 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-08 19:53:50,939 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:53:50,939 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:53:50,939 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,939 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:53:50,941 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,941 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:53:50,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:53:50,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,941 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:53:50,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:53:50,942 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:53:50,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:53:50,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:53:50,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,943 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:53:50,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,946 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,946 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:53:50,946 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,946 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:53:50,946 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,946 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 19:53:50,946 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:53:50,946 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:53:50,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:53:50,946 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:53:50,946 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:50,947 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,947 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,947 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:50,947 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-08 19:53:50,947 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:53:50,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:53:50,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:53:51,047 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:53:51,049 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:53:51,056 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:53:51,058 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-08 19:53:51,058 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:51,058 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 19:53:51,059 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:53:51,059 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-08 19:53:51,059 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-08 19:53:51,060 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:51,060 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:51,062 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:51,062 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:53:51,062 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:53:51,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:53:51,062 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:51,062 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:53:51,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:51,063 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:51,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:53:51,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:51,063 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:51,064 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-08 19:53:51,064 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:51,065 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-08 19:53:51,065 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:53:51,065 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-08 19:53:51,066 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:53:51,066 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:53:51,066 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:53:51,066 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. started... 2023-03-08 19:53:51,066 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 4ad1e88a375de73ef8abab9078c9a436 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:53:51,488 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090791004df1cdfa49558ccd1e3d3473b0dc to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015090791004df1cdfa49558ccd1e3d3473b0dc 2023-03-08 19:53:51,898 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/.tmp/family1/6f324d52bf4346d2850da86667611be2 2023-03-08 19:53:51,906 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/.tmp/family1/6f324d52bf4346d2850da86667611be2 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/6f324d52bf4346d2850da86667611be2 2023-03-08 19:53:51,913 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/6f324d52bf4346d2850da86667611be2, entries=1, sequenceid=37, filesize=5.0 K 2023-03-08 19:53:51,915 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 4ad1e88a375de73ef8abab9078c9a436 in 849ms, sequenceid=37, compaction requested=true 2023-03-08 19:53:51,915 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4ad1e88a375de73ef8abab9078c9a436: 2023-03-08 19:53:51,915 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:53:51,915 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:53:51,915 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:53:51,915 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:51,915 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-08 19:53:51,915 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-08 19:53:51,917 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:51,917 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:51,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:51,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:53:51,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:53:51,918 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:51,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:53:51,918 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:53:51,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:53:51,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:51,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:51,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:53:51,919 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-08 19:53:51,920 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1108016d[Count = 0] remaining members to acquire global barrier 2023-03-08 19:53:51,920 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-08 19:53:51,920 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:51,921 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:51,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:51,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:51,921 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-08 19:53:51,921 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-08 19:53:51,921 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:53:51,922 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:51,922 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:53:51,923 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-08 19:53:51,923 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:51,923 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:53:51,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:51,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:53:51,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:53:51,923 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-08 19:53:51,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:53:51,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:53:51,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:51,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:51,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:53:51,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:51,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:51,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:53:51,926 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-08 19:53:51,926 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-08 19:53:51,926 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:53:51,926 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:53:51,926 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:51,926 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:53:51,930 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:51,930 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:51,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:51,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:51,930 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:53:51,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:51,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:53:51,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:53:51,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:53:51,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:53:51,931 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:51,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:53:51,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:51,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:51,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:53:51,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:51,932 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:51,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:51,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:53:51,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:51,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:51,935 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:53:51,935 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:51,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:53:51,935 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:53:51,936 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 19:53:51,936 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:53:51,935 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:53:51,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:53:51,935 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:51,936 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:51,937 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:51,937 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:51,937 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:51,937 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:51,937 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-08 19:53:51,937 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-08 19:53:51,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:53:51,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:53:51,937 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:53:52,037 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:53:52,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:53:52,047 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:53:52,049 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-08 19:53:52,050 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:52,050 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 19:53:52,050 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:53:52,050 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-08 19:53:52,050 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-08 19:53:52,051 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:52,051 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:52,057 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:52,057 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:53:52,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:53:52,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:53:52,057 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:52,057 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:53:52,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:52,058 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:52,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:53:52,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:52,058 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:52,059 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-08 19:53:52,059 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:52,059 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-08 19:53:52,059 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:53:52,060 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-08 19:53:52,060 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:53:52,060 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:53:52,060 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:53:52,060 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. started... 2023-03-08 19:53:52,060 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 4ad1e88a375de73ef8abab9078c9a436 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:53:52,482 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112097c3170d1c6547b594bc3f01d6c81b36 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015112097c3170d1c6547b594bc3f01d6c81b36 2023-03-08 19:53:52,549 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/.tmp/family1/e7f48e7700624c53a648088bbc121025 2023-03-08 19:53:52,560 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/.tmp/family1/e7f48e7700624c53a648088bbc121025 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/e7f48e7700624c53a648088bbc121025 2023-03-08 19:53:52,568 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/e7f48e7700624c53a648088bbc121025, entries=1, sequenceid=41, filesize=5.0 K 2023-03-08 19:53:52,569 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 4ad1e88a375de73ef8abab9078c9a436 in 509ms, sequenceid=41, compaction requested=true 2023-03-08 19:53:52,569 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4ad1e88a375de73ef8abab9078c9a436: 2023-03-08 19:53:52,569 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:53:52,569 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:53:52,570 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:53:52,570 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:52,570 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-08 19:53:52,570 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-08 19:53:52,571 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:52,571 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:52,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:52,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:53:52,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:53:52,572 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:52,572 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:53:52,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:53:52,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:53:52,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:52,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:52,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:53:52,573 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-08 19:53:52,573 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@39e3675[Count = 0] remaining members to acquire global barrier 2023-03-08 19:53:52,573 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-08 19:53:52,574 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:52,575 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:52,575 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:52,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:52,575 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-08 19:53:52,575 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:52,575 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:53:52,575 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-08 19:53:52,575 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:53:52,580 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:52,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:52,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:53:52,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:53:52,580 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-08 19:53:52,587 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:53:52,587 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-08 19:53:52,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:53:52,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:53:52,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:52,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:52,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:53:52,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:52,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:52,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:53:52,592 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-08 19:53:52,592 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-08 19:53:52,592 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:53:52,592 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:53:52,592 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:52,592 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:53:52,594 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:52,594 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:52,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:52,594 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:53:52,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:52,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:52,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:53:52,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:53:52,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:53:52,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:53:52,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:53:52,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:52,595 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:52,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:52,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:53:52,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:52,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:52,597 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:52,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:53:52,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:52,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:52,603 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:53:52,603 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-08 19:53:52,603 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:53:52,603 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-08 19:53:52,603 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:53:52,603 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:52,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:53:52,604 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:52,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:53:52,604 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:53:52,603 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:53:52,604 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:52,604 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:52,604 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:52,604 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:52,604 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:52,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:53:52,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:53:52,704 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:53:52,706 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:53:52,712 DEBUG [Listener at localhost/34049] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-03-08 19:53:52,715 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53842, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-03-08 19:53:52,717 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39989] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:52,733 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-08 19:53:52,734 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-08 19:53:52,736 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=12, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678305832736 2023-03-08 19:53:52,738 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 19:53:52,740 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 19:53:52,744 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-03-08 19:53:52,744 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-08 19:53:52,744 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] zookeeper.ReadOnlyZKClient(139): Connect 0x11e88228 to 127.0.0.1:56751 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 19:53:52,750 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@50335927, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 19:53:52,756 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 19:53:52,863 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010332e13db965a143c4a582dd4d7ab46e2c to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010332e13db965a143c4a582dd4d7ab46e2c 2023-03-08 19:53:52,865 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-08 19:53:52,888 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 19:53:52,890 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43942, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 19:53:52,903 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 19:53:52,905 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53848, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 19:53:52,942 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/0a62accd92ae43f38540fe5fbeb09006 first=Optional[r3] last=Optional[r6] 2023-03-08 19:53:52,954 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436., hostname=jenkins-hbase4.apache.org,33367,1678305218680, seqNum=2 for row with hfile group [{family1,hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/0a62accd92ae43f38540fe5fbeb09006}] 2023-03-08 19:53:52,971 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(763): Validating hfile at hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/0a62accd92ae43f38540fe5fbeb09006 for inclusion in 4ad1e88a375de73ef8abab9078c9a436/family1 2023-03-08 19:53:52,977 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-08 19:53:52,978 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 19:53:52,978 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(2446): Flush status journal for 4ad1e88a375de73ef8abab9078c9a436: 2023-03-08 19:53:52,979 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/0a62accd92ae43f38540fe5fbeb09006 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__hjptrg3ckaud67rubp8scn10mmf4nojo4cpplarrs4da3s4bbru67f4imu2mff0f/family1/0a62accd92ae43f38540fe5fbeb09006 2023-03-08 19:53:52,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__hjptrg3ckaud67rubp8scn10mmf4nojo4cpplarrs4da3s4bbru67f4imu2mff0f/family1/0a62accd92ae43f38540fe5fbeb09006 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/bff014a5efd8404da869ccc7d9764d27_SeqId_44_ 2023-03-08 19:53:52,983 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(849): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__hjptrg3ckaud67rubp8scn10mmf4nojo4cpplarrs4da3s4bbru67f4imu2mff0f/family1/0a62accd92ae43f38540fe5fbeb09006 into 4ad1e88a375de73ef8abab9078c9a436/family1 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/bff014a5efd8404da869ccc7d9764d27_SeqId_44_ - updating store file list. 2023-03-08 19:53:52,990 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(882): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/bff014a5efd8404da869ccc7d9764d27_SeqId_44_ into 4ad1e88a375de73ef8abab9078c9a436/family1 2023-03-08 19:53:52,990 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(855): Successfully loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__hjptrg3ckaud67rubp8scn10mmf4nojo4cpplarrs4da3s4bbru67f4imu2mff0f/family1/0a62accd92ae43f38540fe5fbeb09006 into 4ad1e88a375de73ef8abab9078c9a436/family1 (new location: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/bff014a5efd8404da869ccc7d9764d27_SeqId_44_) 2023-03-08 19:53:52,990 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__hjptrg3ckaud67rubp8scn10mmf4nojo4cpplarrs4da3s4bbru67f4imu2mff0f/family1/0a62accd92ae43f38540fe5fbeb09006 2023-03-08 19:53:52,992 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-08 19:53:52,998 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51357 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-08 19:53:53,000 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(1912): 4ad1e88a375de73ef8abab9078c9a436/family1 is initiating minor compaction 2023-03-08 19:53:53,001 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@3ce01b95; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 19:53:53,001 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 4ad1e88a375de73ef8abab9078c9a436/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:53:53,002 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/d7bc8921991b4b9e869f5f7ec27724b0, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/27431a5ef048444da16f258fd0926546, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/865e3957809340c1904c484f5006e36b, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/6c4321ddc714451f82661dfe2aad1aac, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/cb9806e2e04b4d8e85ddbd1bc8a679ff, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/75a5720011f34e3da146711c8486e003, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/73f8831bdc3e472c8dd43a897d3f0d03, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/6f324d52bf4346d2850da86667611be2, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/e7f48e7700624c53a648088bbc121025, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/bff014a5efd8404da869ccc7d9764d27_SeqId_44_] into tmpdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/.tmp, totalSize=50.2 K 2023-03-08 19:53:53,003 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting d7bc8921991b4b9e869f5f7ec27724b0, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-08 19:53:53,003 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 27431a5ef048444da16f258fd0926546, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-08 19:53:53,003 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 865e3957809340c1904c484f5006e36b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-08 19:53:53,003 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 6c4321ddc714451f82661dfe2aad1aac, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-08 19:53:53,003 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting cb9806e2e04b4d8e85ddbd1bc8a679ff, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-08 19:53:53,003 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 75a5720011f34e3da146711c8486e003, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-08 19:53:53,003 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 73f8831bdc3e472c8dd43a897d3f0d03, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-08 19:53:53,003 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 6f324d52bf4346d2850da86667611be2, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-08 19:53:53,003 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting e7f48e7700624c53a648088bbc121025, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-08 19:53:53,003 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting bff014a5efd8404da869ccc7d9764d27_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-08 19:53:53,018 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-08 19:53:53,020 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:53:53,026 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512284c660201c5ac4c859aa1e35c2f397165 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512284c660201c5ac4c859aa1e35c2f397165 2023-03-08 19:53:53,027 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512316ddba1fa3b0a4f6da5c6261316cf19f4 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512316ddba1fa3b0a4f6da5c6261316cf19f4 2023-03-08 19:53:53,030 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601016d1cf375f3664352be78a22a9de8b418 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601016d1cf375f3664352be78a22a9de8b418 2023-03-08 19:53:53,033 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010367ee4bebfce54c8ebd6ad68637323bda to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010367ee4bebfce54c8ebd6ad68637323bda 2023-03-08 19:53:53,033 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 4 expired mob files are deleted 2023-03-08 19:53:53,033 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-08 19:53:53,034 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 19:53:53,034 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 19:53:53,034 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 19:53:53,034 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 19:53:53,034 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 19:53:53,034 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 19:53:53,035 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 19:53:53,035 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 19:53:53,057 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] throttle.PressureAwareThroughputController(145): 4ad1e88a375de73ef8abab9078c9a436#family1#compaction#10 average throughput is 0.48 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 19:53:53,482 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/.tmp/family1/9fbea280641d4791be976f96a309df28 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/9fbea280641d4791be976f96a309df28 2023-03-08 19:53:53,496 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 4ad1e88a375de73ef8abab9078c9a436/family1 of 4ad1e88a375de73ef8abab9078c9a436 into 9fbea280641d4791be976f96a309df28(size=6.6 K), total size for store is 182.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 19:53:53,497 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 4ad1e88a375de73ef8abab9078c9a436: 2023-03-08 19:53:53,498 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436., storeName=family1, fileCount=10, fileSize=50.2 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.5 K), priority=2, time=1678305232999; duration=0sec 2023-03-08 19:53:53,499 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 19:53:53,520 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/d7bc8921991b4b9e869f5f7ec27724b0, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/27431a5ef048444da16f258fd0926546, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/865e3957809340c1904c484f5006e36b, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/6c4321ddc714451f82661dfe2aad1aac, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/cb9806e2e04b4d8e85ddbd1bc8a679ff, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/75a5720011f34e3da146711c8486e003, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/73f8831bdc3e472c8dd43a897d3f0d03, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/6f324d52bf4346d2850da86667611be2, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/e7f48e7700624c53a648088bbc121025, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/bff014a5efd8404da869ccc7d9764d27_SeqId_44_] to archive 2023-03-08 19:53:53,522 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:53:53,525 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/d7bc8921991b4b9e869f5f7ec27724b0 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/d7bc8921991b4b9e869f5f7ec27724b0 2023-03-08 19:53:53,527 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/27431a5ef048444da16f258fd0926546 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/27431a5ef048444da16f258fd0926546 2023-03-08 19:53:53,529 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/865e3957809340c1904c484f5006e36b to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/865e3957809340c1904c484f5006e36b 2023-03-08 19:53:53,530 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/6c4321ddc714451f82661dfe2aad1aac to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/6c4321ddc714451f82661dfe2aad1aac 2023-03-08 19:53:53,532 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/cb9806e2e04b4d8e85ddbd1bc8a679ff to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/cb9806e2e04b4d8e85ddbd1bc8a679ff 2023-03-08 19:53:53,533 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/75a5720011f34e3da146711c8486e003 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/75a5720011f34e3da146711c8486e003 2023-03-08 19:53:53,535 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/73f8831bdc3e472c8dd43a897d3f0d03 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/73f8831bdc3e472c8dd43a897d3f0d03 2023-03-08 19:53:53,536 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/6f324d52bf4346d2850da86667611be2 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/6f324d52bf4346d2850da86667611be2 2023-03-08 19:53:53,538 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/e7f48e7700624c53a648088bbc121025 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/e7f48e7700624c53a648088bbc121025 2023-03-08 19:53:53,539 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/bff014a5efd8404da869ccc7d9764d27_SeqId_44_ to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/bff014a5efd8404da869ccc7d9764d27_SeqId_44_ 2023-03-08 19:53:53,921 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205cdf87d5b23464971b2a44638c5baab12 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205cdf87d5b23464971b2a44638c5baab12 2023-03-08 19:53:53,923 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-08 19:53:53,953 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/2899017e9bd54049a5590a6924ede80c first=Optional[r00] last=Optional[r2] 2023-03-08 19:53:53,954 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436., hostname=jenkins-hbase4.apache.org,33367,1678305218680, seqNum=2 for row with hfile group [{family1,hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/2899017e9bd54049a5590a6924ede80c}] 2023-03-08 19:53:53,956 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(763): Validating hfile at hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/2899017e9bd54049a5590a6924ede80c for inclusion in 4ad1e88a375de73ef8abab9078c9a436/family1 2023-03-08 19:53:53,960 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-08 19:53:53,961 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 19:53:53,961 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(2446): Flush status journal for 4ad1e88a375de73ef8abab9078c9a436: 2023-03-08 19:53:53,962 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/2899017e9bd54049a5590a6924ede80c to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__9uqm7dre1dme4t2ph5u5h7gbp9957o83gfjndboa8fbl4parlgalkehu9372cvac/family1/2899017e9bd54049a5590a6924ede80c 2023-03-08 19:53:53,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__9uqm7dre1dme4t2ph5u5h7gbp9957o83gfjndboa8fbl4parlgalkehu9372cvac/family1/2899017e9bd54049a5590a6924ede80c as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/7f967acf4e8e4ef999395524eb1b8ff8_SeqId_47_ 2023-03-08 19:53:53,965 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(849): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__9uqm7dre1dme4t2ph5u5h7gbp9957o83gfjndboa8fbl4parlgalkehu9372cvac/family1/2899017e9bd54049a5590a6924ede80c into 4ad1e88a375de73ef8abab9078c9a436/family1 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/7f967acf4e8e4ef999395524eb1b8ff8_SeqId_47_ - updating store file list. 2023-03-08 19:53:53,971 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(882): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/7f967acf4e8e4ef999395524eb1b8ff8_SeqId_47_ into 4ad1e88a375de73ef8abab9078c9a436/family1 2023-03-08 19:53:53,971 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(855): Successfully loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__9uqm7dre1dme4t2ph5u5h7gbp9957o83gfjndboa8fbl4parlgalkehu9372cvac/family1/2899017e9bd54049a5590a6924ede80c into 4ad1e88a375de73ef8abab9078c9a436/family1 (new location: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/7f967acf4e8e4ef999395524eb1b8ff8_SeqId_47_) 2023-03-08 19:53:53,971 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__9uqm7dre1dme4t2ph5u5h7gbp9957o83gfjndboa8fbl4parlgalkehu9372cvac/family1/2899017e9bd54049a5590a6924ede80c 2023-03-08 19:53:53,971 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-08 19:53:53,973 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 367249 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-08 19:53:53,973 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(1912): 4ad1e88a375de73ef8abab9078c9a436/family1 is initiating minor compaction (all files) 2023-03-08 19:53:53,973 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@3d033f51; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 19:53:53,973 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 4ad1e88a375de73ef8abab9078c9a436/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:53:53,974 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/0435e74f700e4bf3a70e98f4689f8070, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/9fbea280641d4791be976f96a309df28, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/7f967acf4e8e4ef999395524eb1b8ff8_SeqId_47_] into tmpdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/.tmp, totalSize=358.6 K 2023-03-08 19:53:53,974 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 0435e74f700e4bf3a70e98f4689f8070, keycount=1000, bloomtype=ROW, size=175.8 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878843488 2023-03-08 19:53:53,975 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 9fbea280641d4791be976f96a309df28, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621243488 2023-03-08 19:53:53,975 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 7f967acf4e8e4ef999395524eb1b8ff8_SeqId_47_, keycount=1002, bloomtype=ROW, size=176.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878843488 2023-03-08 19:53:53,976 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-08 19:53:53,977 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:53:53,979 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511300f5b82cfb5ce41389fa3db34e33c098d to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511300f5b82cfb5ce41389fa3db34e33c098d 2023-03-08 19:53:53,981 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201faa8f7bf04cd4d08bc3dd154ed269526 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201faa8f7bf04cd4d08bc3dd154ed269526 2023-03-08 19:53:53,982 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512055640501452aa45cbaaf465648d56e3ad to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512055640501452aa45cbaaf465648d56e3ad 2023-03-08 19:53:53,982 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 3 expired mob files are deleted 2023-03-08 19:53:53,982 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-08 19:53:53,983 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 19:53:53,983 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-08 19:53:53,984 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 19:53:53,985 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=12, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-08 19:53:53,987 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 1.2620 sec 2023-03-08 19:53:54,012 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] throttle.PressureAwareThroughputController(145): 4ad1e88a375de73ef8abab9078c9a436#family1#compaction#11 average throughput is 7.18 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 19:53:54,135 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthly from 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-08 19:53:54,146 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure2.ProcedureExecutor(1029): Stored pid=13, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:54,162 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-03-08 19:53:54,164 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-03-08 19:53:54,164 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-03-08 19:53:54,168 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly}] 2023-03-08 19:53:54,182 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=4ad1e88a375de73ef8abab9078c9a436, REOPEN/MOVE}] 2023-03-08 19:53:54,183 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=4ad1e88a375de73ef8abab9078c9a436, REOPEN/MOVE 2023-03-08 19:53:54,184 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=4ad1e88a375de73ef8abab9078c9a436, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:54,184 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678305234184"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678305234184"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678305234184"}]},"ts":"1678305234184"} 2023-03-08 19:53:54,187 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; CloseRegionProcedure 4ad1e88a375de73ef8abab9078c9a436, server=jenkins-hbase4.apache.org,33367,1678305218680}] 2023-03-08 19:53:54,346 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 4ad1e88a375de73ef8abab9078c9a436 2023-03-08 19:53:54,346 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 4ad1e88a375de73ef8abab9078c9a436, disabling compactions & flushes 2023-03-08 19:53:54,346 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:53:54,846 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/.tmp/family1/52767f06a80447bfb3abae464db4e804 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/52767f06a80447bfb3abae464db4e804 2023-03-08 19:53:54,853 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 4ad1e88a375de73ef8abab9078c9a436/family1 of 4ad1e88a375de73ef8abab9078c9a436 into 52767f06a80447bfb3abae464db4e804(size=177.2 K), total size for store is 177.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 19:53:54,853 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 4ad1e88a375de73ef8abab9078c9a436: 2023-03-08 19:53:54,853 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:53:54,854 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436., storeName=family1, fileCount=3, fileSize=358.6 K (175.8 K, 6.6 K, 176.2 K), priority=2, time=1678305233973; duration=0sec 2023-03-08 19:53:54,854 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:53:54,854 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 19:53:54,854 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. after waiting 0 ms 2023-03-08 19:53:54,854 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:53:54,857 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/0435e74f700e4bf3a70e98f4689f8070, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/9fbea280641d4791be976f96a309df28, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/7f967acf4e8e4ef999395524eb1b8ff8_SeqId_47_] to archive 2023-03-08 19:53:54,858 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:53:54,860 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/0435e74f700e4bf3a70e98f4689f8070 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/0435e74f700e4bf3a70e98f4689f8070 2023-03-08 19:53:54,861 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/9fbea280641d4791be976f96a309df28 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/9fbea280641d4791be976f96a309df28 2023-03-08 19:53:54,863 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/7f967acf4e8e4ef999395524eb1b8ff8_SeqId_47_ to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/7f967acf4e8e4ef999395524eb1b8ff8_SeqId_47_ 2023-03-08 19:53:54,870 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-08 19:53:54,872 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:53:54,872 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 4ad1e88a375de73ef8abab9078c9a436: 2023-03-08 19:53:54,872 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 4ad1e88a375de73ef8abab9078c9a436 to self. 2023-03-08 19:53:54,874 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 4ad1e88a375de73ef8abab9078c9a436 2023-03-08 19:53:54,874 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=4ad1e88a375de73ef8abab9078c9a436, regionState=CLOSED 2023-03-08 19:53:54,875 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678305234874"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678305234874"}]},"ts":"1678305234874"} 2023-03-08 19:53:54,879 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=15 2023-03-08 19:53:54,879 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=15, state=SUCCESS; CloseRegionProcedure 4ad1e88a375de73ef8abab9078c9a436, server=jenkins-hbase4.apache.org,33367,1678305218680 in 689 msec 2023-03-08 19:53:54,879 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=4ad1e88a375de73ef8abab9078c9a436, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,33367,1678305218680; forceNewPlan=false, retain=true 2023-03-08 19:53:55,030 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=4ad1e88a375de73ef8abab9078c9a436, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:55,030 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678305235030"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678305235030"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678305235030"}]},"ts":"1678305235030"} 2023-03-08 19:53:55,033 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure 4ad1e88a375de73ef8abab9078c9a436, server=jenkins-hbase4.apache.org,33367,1678305218680}] 2023-03-08 19:53:55,189 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:53:55,189 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 4ad1e88a375de73ef8abab9078c9a436, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436.', STARTKEY => '', ENDKEY => ''} 2023-03-08 19:53:55,189 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 4ad1e88a375de73ef8abab9078c9a436 2023-03-08 19:53:55,190 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:53:55,190 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 4ad1e88a375de73ef8abab9078c9a436 2023-03-08 19:53:55,190 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 4ad1e88a375de73ef8abab9078c9a436 2023-03-08 19:53:55,191 INFO [StoreOpener-4ad1e88a375de73ef8abab9078c9a436-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 4ad1e88a375de73ef8abab9078c9a436 2023-03-08 19:53:55,192 DEBUG [StoreOpener-4ad1e88a375de73ef8abab9078c9a436-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1 2023-03-08 19:53:55,192 DEBUG [StoreOpener-4ad1e88a375de73ef8abab9078c9a436-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1 2023-03-08 19:53:55,193 INFO [StoreOpener-4ad1e88a375de73ef8abab9078c9a436-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4ad1e88a375de73ef8abab9078c9a436 columnFamilyName family1 2023-03-08 19:53:55,200 DEBUG [StoreOpener-4ad1e88a375de73ef8abab9078c9a436-1] regionserver.HStore(539): loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/52767f06a80447bfb3abae464db4e804 2023-03-08 19:53:55,200 INFO [StoreOpener-4ad1e88a375de73ef8abab9078c9a436-1] regionserver.HStore(310): Store=4ad1e88a375de73ef8abab9078c9a436/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:53:55,201 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436 2023-03-08 19:53:55,203 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436 2023-03-08 19:53:55,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 4ad1e88a375de73ef8abab9078c9a436 2023-03-08 19:53:55,206 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 4ad1e88a375de73ef8abab9078c9a436; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71120200, jitterRate=0.05977356433868408}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 19:53:55,206 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 4ad1e88a375de73ef8abab9078c9a436: 2023-03-08 19:53:55,207 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436., pid=17, masterSystemTime=1678305235185 2023-03-08 19:53:55,210 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:53:55,210 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:53:55,210 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=4ad1e88a375de73ef8abab9078c9a436, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:55,210 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678305235210"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678305235210"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678305235210"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678305235210"}]},"ts":"1678305235210"} 2023-03-08 19:53:55,214 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-03-08 19:53:55,214 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure 4ad1e88a375de73ef8abab9078c9a436, server=jenkins-hbase4.apache.org,33367,1678305218680 in 179 msec 2023-03-08 19:53:55,216 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=14 2023-03-08 19:53:55,216 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=14, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=4ad1e88a375de73ef8abab9078c9a436, REOPEN/MOVE in 1.0320 sec 2023-03-08 19:53:55,219 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-03-08 19:53:55,220 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.0500 sec 2023-03-08 19:53:55,222 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=13, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.0810 sec 2023-03-08 19:53:55,225 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=13 2023-03-08 19:53:55,231 INFO [Listener at localhost/34049] compactions.TestMobCompactor(278): alter status finished 2023-03-08 19:53:55,232 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39989] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-08 19:53:55,234 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-08 19:53:55,235 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-08 19:53:55,236 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 19:53:55,236 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=18, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678305835236 2023-03-08 19:53:55,237 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 19:53:55,237 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-03-08 19:53:55,237 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-08 19:53:55,238 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] zookeeper.ReadOnlyZKClient(139): Connect 0x72d632fe to 127.0.0.1:56751 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 19:53:55,245 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@62d9afa8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 19:53:55,245 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 19:53:55,245 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 19:53:55,245 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 19:53:55,245 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 19:53:55,246 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 19:53:56,089 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511285a5405682cff407297d375c1516dc85f to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511285a5405682cff407297d375c1516dc85f 2023-03-08 19:53:56,091 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-08 19:53:56,110 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 19:53:56,112 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52412, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 19:53:56,131 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 19:53:56,133 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57348, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 19:53:56,146 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/956b646fe8aa47fab993c11cfb4b718b first=Optional[r05] last=Optional[r8] 2023-03-08 19:53:56,147 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436., hostname=jenkins-hbase4.apache.org,33367,1678305218680, seqNum=51 for row with hfile group [{family1,hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/956b646fe8aa47fab993c11cfb4b718b}] 2023-03-08 19:53:56,153 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(763): Validating hfile at hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/956b646fe8aa47fab993c11cfb4b718b for inclusion in 4ad1e88a375de73ef8abab9078c9a436/family1 2023-03-08 19:53:56,157 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-03-08 19:53:56,157 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 19:53:56,157 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(2446): Flush status journal for 4ad1e88a375de73ef8abab9078c9a436: 2023-03-08 19:53:56,159 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/956b646fe8aa47fab993c11cfb4b718b to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__mbttkvrlgth6enbq3onheq921if7nnsbkkoub42u3gr4kkku7ehmvn1hlkat07dk/family1/956b646fe8aa47fab993c11cfb4b718b 2023-03-08 19:53:56,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__mbttkvrlgth6enbq3onheq921if7nnsbkkoub42u3gr4kkku7ehmvn1hlkat07dk/family1/956b646fe8aa47fab993c11cfb4b718b as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/10d1c6c4a0fb4f1d8ced2160751db694_SeqId_53_ 2023-03-08 19:53:56,163 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(849): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__mbttkvrlgth6enbq3onheq921if7nnsbkkoub42u3gr4kkku7ehmvn1hlkat07dk/family1/956b646fe8aa47fab993c11cfb4b718b into 4ad1e88a375de73ef8abab9078c9a436/family1 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/10d1c6c4a0fb4f1d8ced2160751db694_SeqId_53_ - updating store file list. 2023-03-08 19:53:56,173 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(882): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/10d1c6c4a0fb4f1d8ced2160751db694_SeqId_53_ into 4ad1e88a375de73ef8abab9078c9a436/family1 2023-03-08 19:53:56,173 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(855): Successfully loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__mbttkvrlgth6enbq3onheq921if7nnsbkkoub42u3gr4kkku7ehmvn1hlkat07dk/family1/956b646fe8aa47fab993c11cfb4b718b into 4ad1e88a375de73ef8abab9078c9a436/family1 (new location: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/family1/10d1c6c4a0fb4f1d8ced2160751db694_SeqId_53_) 2023-03-08 19:53:56,174 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__mbttkvrlgth6enbq3onheq921if7nnsbkkoub42u3gr4kkku7ehmvn1hlkat07dk/family1/956b646fe8aa47fab993c11cfb4b718b 2023-03-08 19:53:56,176 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-08 19:53:56,177 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:53:56,179 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015112097c3170d1c6547b594bc3f01d6c81b36 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015112097c3170d1c6547b594bc3f01d6c81b36 2023-03-08 19:53:56,180 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015112885b1d054e50f4fbab508f15b87ee94db to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015112885b1d054e50f4fbab508f15b87ee94db 2023-03-08 19:53:56,180 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 2 expired mob files are deleted 2023-03-08 19:53:56,180 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-08 19:53:56,181 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 19:53:56,181 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 19:53:56,181 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-08 19:53:56,181 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-08 19:53:56,181 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 19:53:56,182 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=18, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-08 19:53:56,185 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 950 msec 2023-03-08 19:53:56,278 INFO [Listener at localhost/34049] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=248 (was 211) Potentially hanging thread: (jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56751@0x11e88228-SendThread(127.0.0.1:56751) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56751@0x72d632fe-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56751@0x72d632fe-SendThread(127.0.0.1:56751) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-305148072_15 at /127.0.0.1:48252 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: htable-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_968202085_15 at /127.0.0.1:46134 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-305148072_15 at /127.0.0.1:46220 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-305148072_15 at /127.0.0.1:46260 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-305148072_15 at /127.0.0.1:46148 [Waiting for operation #23] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-305148072_15 at /127.0.0.1:46208 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-305148072_15 at /127.0.0.1:46132 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6f47bed9-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-305148072_15 at /127.0.0.1:46118 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-305148072_15 at /127.0.0.1:45994 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56751@0x11e88228 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/512641589.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6f47bed9-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-305148072_15 at /127.0.0.1:45980 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: MobCompactor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-305148072_15 at /127.0.0.1:46192 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-305148072_15 at /127.0.0.1:46098 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x58ee3f91-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x58ee3f91-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase4:33367-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-305148072_15 at /127.0.0.1:46026 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-305148072_15 at /127.0.0.1:46060 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-305148072_15 at /127.0.0.1:46258 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56751@0x72d632fe sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/512641589.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-305148072_15 at /127.0.0.1:46206 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56751@0x11e88228-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-305148072_15 at /127.0.0.1:46236 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6f47bed9-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) - Thread LEAK? -, OpenFileDescriptor=598 (was 475) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=295 (was 330), ProcessCount=170 (was 170), AvailableMemoryMB=8239 (was 8590) 2023-03-08 19:53:56,292 INFO [Listener at localhost/34049] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=248, OpenFileDescriptor=598, MaxFileDescriptor=60000, SystemLoadAverage=295, ProcessCount=170, AvailableMemoryMB=8238 2023-03-08 19:53:56,294 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-08 19:53:56,296 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:56,297 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 19:53:56,298 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithMonthlyPolicy" procId is: 19 2023-03-08 19:53:56,298 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 19:53:56,299 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-08 19:53:56,300 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0 2023-03-08 19:53:56,301 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0 empty. 2023-03-08 19:53:56,301 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0 2023-03-08 19:53:56,301 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithMonthlyPolicy regions 2023-03-08 19:53:56,400 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-08 19:53:56,601 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-08 19:53:56,713 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-08 19:53:56,714 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => 3551c80f1cc0fa597fb1a918e4b1b6f0, NAME => 'testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp 2023-03-08 19:53:56,902 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-08 19:53:57,122 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:53:57,122 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1604): Closing 3551c80f1cc0fa597fb1a918e4b1b6f0, disabling compactions & flushes 2023-03-08 19:53:57,122 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. 2023-03-08 19:53:57,122 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. 2023-03-08 19:53:57,122 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. after waiting 0 ms 2023-03-08 19:53:57,122 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. 2023-03-08 19:53:57,122 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. 2023-03-08 19:53:57,122 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for 3551c80f1cc0fa597fb1a918e4b1b6f0: 2023-03-08 19:53:57,127 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 19:53:57,128 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678305237128"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678305237128"}]},"ts":"1678305237128"} 2023-03-08 19:53:57,130 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-08 19:53:57,132 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 19:53:57,132 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678305237132"}]},"ts":"1678305237132"} 2023-03-08 19:53:57,134 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-03-08 19:53:57,138 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=3551c80f1cc0fa597fb1a918e4b1b6f0, ASSIGN}] 2023-03-08 19:53:57,140 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=3551c80f1cc0fa597fb1a918e4b1b6f0, ASSIGN 2023-03-08 19:53:57,141 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=3551c80f1cc0fa597fb1a918e4b1b6f0, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33367,1678305218680; forceNewPlan=false, retain=false 2023-03-08 19:53:57,292 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=3551c80f1cc0fa597fb1a918e4b1b6f0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:57,292 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678305237292"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678305237292"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678305237292"}]},"ts":"1678305237292"} 2023-03-08 19:53:57,294 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure 3551c80f1cc0fa597fb1a918e4b1b6f0, server=jenkins-hbase4.apache.org,33367,1678305218680}] 2023-03-08 19:53:57,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-08 19:53:57,450 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. 2023-03-08 19:53:57,450 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 3551c80f1cc0fa597fb1a918e4b1b6f0, NAME => 'testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0.', STARTKEY => '', ENDKEY => ''} 2023-03-08 19:53:57,451 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithMonthlyPolicy 3551c80f1cc0fa597fb1a918e4b1b6f0 2023-03-08 19:53:57,451 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:53:57,451 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 3551c80f1cc0fa597fb1a918e4b1b6f0 2023-03-08 19:53:57,451 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 3551c80f1cc0fa597fb1a918e4b1b6f0 2023-03-08 19:53:57,452 INFO [StoreOpener-3551c80f1cc0fa597fb1a918e4b1b6f0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 3551c80f1cc0fa597fb1a918e4b1b6f0 2023-03-08 19:53:57,454 DEBUG [StoreOpener-3551c80f1cc0fa597fb1a918e4b1b6f0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1 2023-03-08 19:53:57,454 DEBUG [StoreOpener-3551c80f1cc0fa597fb1a918e4b1b6f0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1 2023-03-08 19:53:57,454 INFO [StoreOpener-3551c80f1cc0fa597fb1a918e4b1b6f0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3551c80f1cc0fa597fb1a918e4b1b6f0 columnFamilyName family1 2023-03-08 19:53:57,456 INFO [StoreOpener-3551c80f1cc0fa597fb1a918e4b1b6f0-1] regionserver.HStore(310): Store=3551c80f1cc0fa597fb1a918e4b1b6f0/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:53:57,457 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0 2023-03-08 19:53:57,458 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0 2023-03-08 19:53:57,461 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 3551c80f1cc0fa597fb1a918e4b1b6f0 2023-03-08 19:53:57,463 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 19:53:57,463 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 3551c80f1cc0fa597fb1a918e4b1b6f0; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62514794, jitterRate=-0.0684569776058197}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 19:53:57,463 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 3551c80f1cc0fa597fb1a918e4b1b6f0: 2023-03-08 19:53:57,464 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0., pid=21, masterSystemTime=1678305237447 2023-03-08 19:53:57,466 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. 2023-03-08 19:53:57,466 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. 2023-03-08 19:53:57,467 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=3551c80f1cc0fa597fb1a918e4b1b6f0, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:57,467 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678305237466"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678305237466"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678305237466"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678305237466"}]},"ts":"1678305237466"} 2023-03-08 19:53:57,471 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=20 2023-03-08 19:53:57,471 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure 3551c80f1cc0fa597fb1a918e4b1b6f0, server=jenkins-hbase4.apache.org,33367,1678305218680 in 175 msec 2023-03-08 19:53:57,473 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-03-08 19:53:57,473 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=3551c80f1cc0fa597fb1a918e4b1b6f0, ASSIGN in 333 msec 2023-03-08 19:53:57,474 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 19:53:57,474 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678305237474"}]},"ts":"1678305237474"} 2023-03-08 19:53:57,476 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-03-08 19:53:57,479 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 19:53:57,481 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy in 1.1850 sec 2023-03-08 19:53:58,405 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-08 19:53:58,406 INFO [Listener at localhost/34049] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithMonthlyPolicy, procId: 19 completed 2023-03-08 19:53:58,453 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:53:58,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:58,455 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 19:53:58,455 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:53:58,456 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 19:53:58,456 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 19:53:58,457 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:58,457 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:58,458 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:58,458 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:53:58,458 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:53:58,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:53:58,459 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:58,459 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:53:58,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:58,460 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:58,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:53:58,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:58,460 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:58,460 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:58,460 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 19:53:58,460 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:53:58,461 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 19:53:58,461 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:53:58,461 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:53:58,461 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. 2023-03-08 19:53:58,461 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. started... 2023-03-08 19:53:58,462 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3551c80f1cc0fa597fb1a918e4b1b6f0 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-08 19:53:58,497 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511300a95ab5edc234e8f8453cfb8c755eb23 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511300a95ab5edc234e8f8453cfb8c755eb23 2023-03-08 19:53:58,902 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/.tmp/family1/4be98f24a4ef4eb6803e1ce815aa33e8 2023-03-08 19:53:58,909 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/.tmp/family1/4be98f24a4ef4eb6803e1ce815aa33e8 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/4be98f24a4ef4eb6803e1ce815aa33e8 2023-03-08 19:53:58,914 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/4be98f24a4ef4eb6803e1ce815aa33e8, entries=1000, sequenceid=5, filesize=166.0 K 2023-03-08 19:53:58,915 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 3551c80f1cc0fa597fb1a918e4b1b6f0 in 453ms, sequenceid=5, compaction requested=false 2023-03-08 19:53:58,916 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 19:53:58,916 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3551c80f1cc0fa597fb1a918e4b1b6f0: 2023-03-08 19:53:58,916 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. 2023-03-08 19:53:58,916 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:53:58,917 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:53:58,917 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:58,917 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 19:53:58,917 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-08 19:53:58,918 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:58,918 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:58,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:58,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:53:58,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:53:58,919 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:58,919 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:53:58,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:53:58,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:53:58,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:58,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:58,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:53:58,920 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 19:53:58,920 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@34017db5[Count = 0] remaining members to acquire global barrier 2023-03-08 19:53:58,920 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 19:53:58,920 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:58,922 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:58,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:58,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:58,923 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 19:53:58,923 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-08 19:53:58,923 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:53:58,923 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:58,923 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:53:58,924 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 19:53:58,924 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:58,924 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:53:58,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:58,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:53:58,924 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-08 19:53:58,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:53:58,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:53:58,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:53:58,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:58,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:58,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:53:58,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:58,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:58,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:53:58,928 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 19:53:58,928 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-08 19:53:58,928 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:53:58,928 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:53:58,928 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:58,928 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:53:58,929 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:58,929 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:58,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:58,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:58,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:53:58,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:53:58,930 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:53:58,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:58,930 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:58,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:53:58,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:53:58,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:53:58,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:58,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:58,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:53:58,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:58,931 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:58,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:58,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:53:58,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:58,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:58,934 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:58,934 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:53:58,934 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:58,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:53:58,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:53:58,934 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 19:53:58,934 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:53:58,934 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:58,934 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:53:58,934 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:58,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:53:58,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:53:58,934 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:53:58,934 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:58,934 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 19:53:58,934 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 19:53:58,934 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:58,935 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:53:58,935 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:59,035 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:53:59,036 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:53:59,042 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:53:59,043 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 19:53:59,044 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:59,044 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 19:53:59,044 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:53:59,044 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 19:53:59,044 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 19:53:59,045 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:59,045 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:59,051 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:59,051 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:53:59,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:53:59,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:53:59,052 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:59,052 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:53:59,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:59,052 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:59,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:53:59,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:59,052 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:59,052 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 19:53:59,053 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:59,053 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 19:53:59,053 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:53:59,053 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 19:53:59,053 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:53:59,053 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:53:59,053 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. 2023-03-08 19:53:59,053 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. started... 2023-03-08 19:53:59,054 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3551c80f1cc0fa597fb1a918e4b1b6f0 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-08 19:53:59,469 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511282219db87b90a443e82956787b41d4d4e to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511282219db87b90a443e82956787b41d4d4e 2023-03-08 19:53:59,875 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/.tmp/family1/73e2073e030d46a78276019b6f4cdda4 2023-03-08 19:53:59,882 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/.tmp/family1/73e2073e030d46a78276019b6f4cdda4 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/73e2073e030d46a78276019b6f4cdda4 2023-03-08 19:53:59,888 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/73e2073e030d46a78276019b6f4cdda4, entries=1, sequenceid=9, filesize=5.0 K 2023-03-08 19:53:59,889 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 3551c80f1cc0fa597fb1a918e4b1b6f0 in 835ms, sequenceid=9, compaction requested=false 2023-03-08 19:53:59,889 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3551c80f1cc0fa597fb1a918e4b1b6f0: 2023-03-08 19:53:59,889 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. 2023-03-08 19:53:59,890 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:53:59,890 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:53:59,890 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:59,890 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 19:53:59,890 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-08 19:53:59,927 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:59,927 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:59,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:59,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:53:59,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:53:59,928 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:59,928 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:53:59,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:53:59,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:53:59,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:59,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:59,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:53:59,930 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 19:53:59,930 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@207a3da3[Count = 0] remaining members to acquire global barrier 2023-03-08 19:53:59,930 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 19:53:59,930 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:59,974 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:59,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:59,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:59,974 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 19:53:59,974 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-08 19:53:59,974 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:59,974 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:53:59,974 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:53:59,997 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 19:53:59,997 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:59,997 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:53:59,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:59,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:53:59,997 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-08 19:53:59,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:53:59,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:53:59,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:53:59,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:59,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:53:59,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:53:59,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:53:59,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:00,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:00,000 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 19:54:00,000 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-08 19:54:00,000 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:00,000 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:00,000 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,000 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:00,027 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,027 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,028 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:00,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:00,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:00,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:00,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:00,028 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:00,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:00,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:00,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,030 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:00,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:00,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:00,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:00,243 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:00,243 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:00,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:00,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:00,243 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:00,243 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 19:54:00,244 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:00,243 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,243 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:00,244 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 19:54:00,244 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,244 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:00,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:00,244 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:00,244 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:00,245 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,245 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,245 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,245 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:00,345 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:00,346 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:00,351 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:00,353 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:00,353 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,353 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 19:54:00,353 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:00,354 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 19:54:00,354 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 19:54:00,354 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,355 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,408 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 19:54:00,436 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:00,436 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:00,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:00,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:00,436 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:00,436 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:00,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,437 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:00,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,437 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,456 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 19:54:00,456 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,456 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 19:54:00,456 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:00,457 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 19:54:00,457 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:00,457 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:00,457 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. 2023-03-08 19:54:00,457 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. started... 2023-03-08 19:54:00,457 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3551c80f1cc0fa597fb1a918e4b1b6f0 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:54:00,483 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120176e8ffa363a141bdb010303788de0801 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120176e8ffa363a141bdb010303788de0801 2023-03-08 19:54:00,889 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/.tmp/family1/c0175e335d6a465393aa5af881be82e3 2023-03-08 19:54:00,897 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/.tmp/family1/c0175e335d6a465393aa5af881be82e3 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/c0175e335d6a465393aa5af881be82e3 2023-03-08 19:54:00,903 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/c0175e335d6a465393aa5af881be82e3, entries=1, sequenceid=13, filesize=4.9 K 2023-03-08 19:54:00,904 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 3551c80f1cc0fa597fb1a918e4b1b6f0 in 447ms, sequenceid=13, compaction requested=true 2023-03-08 19:54:00,904 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3551c80f1cc0fa597fb1a918e4b1b6f0: 2023-03-08 19:54:00,904 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. 2023-03-08 19:54:00,904 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:00,904 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:00,904 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:00,904 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 19:54:00,904 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-08 19:54:00,906 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:00,906 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:00,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:00,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:00,906 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,907 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:00,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:00,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:00,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:00,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:00,908 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 19:54:00,908 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@712843a8[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:00,908 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 19:54:00,908 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,910 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,910 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,911 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 19:54:00,911 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:00,911 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:00,911 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-08 19:54:00,911 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:00,912 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 19:54:00,912 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:00,912 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:00,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:00,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:00,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:00,913 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-08 19:54:00,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:00,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:00,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:00,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:00,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:00,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:00,923 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 19:54:00,923 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-08 19:54:00,923 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:00,923 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:00,923 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,923 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:00,926 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,926 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:00,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:00,926 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,926 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:00,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:00,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:00,927 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:00,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:00,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:00,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:00,928 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:00,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:00,931 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:00,931 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:00,932 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:00,932 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,931 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:00,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:00,932 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,932 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:00,932 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,932 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,932 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:00,932 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:00,932 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 19:54:00,933 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:00,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:00,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:00,933 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:00,933 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:00,933 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 19:54:01,034 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:01,035 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:01,040 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:01,042 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:01,042 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:01,042 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 19:54:01,042 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:01,043 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 19:54:01,043 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 19:54:01,043 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:01,043 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:01,045 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:01,045 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:01,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:01,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:01,045 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:01,045 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:01,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:01,046 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:01,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:01,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:01,046 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:01,051 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 19:54:01,051 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:01,051 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 19:54:01,051 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:01,051 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 19:54:01,052 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:01,052 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:01,052 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. 2023-03-08 19:54:01,052 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. started... 2023-03-08 19:54:01,052 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3551c80f1cc0fa597fb1a918e4b1b6f0 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:54:01,467 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205ae2385370f3e4ed683a6dc90374af25c to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205ae2385370f3e4ed683a6dc90374af25c 2023-03-08 19:54:01,874 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/.tmp/family1/da978af043f947a9abe4c8afd05ed840 2023-03-08 19:54:01,881 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/.tmp/family1/da978af043f947a9abe4c8afd05ed840 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/da978af043f947a9abe4c8afd05ed840 2023-03-08 19:54:01,888 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/da978af043f947a9abe4c8afd05ed840, entries=1, sequenceid=17, filesize=4.9 K 2023-03-08 19:54:01,889 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 3551c80f1cc0fa597fb1a918e4b1b6f0 in 837ms, sequenceid=17, compaction requested=true 2023-03-08 19:54:01,889 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3551c80f1cc0fa597fb1a918e4b1b6f0: 2023-03-08 19:54:01,890 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. 2023-03-08 19:54:01,890 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:01,890 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:01,890 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:01,890 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 19:54:01,890 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-08 19:54:01,892 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:01,892 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:01,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:01,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:01,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:01,892 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:01,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:01,892 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:01,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:01,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:01,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:01,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:01,894 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 19:54:01,894 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3f7ee66[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:01,894 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 19:54:01,894 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:01,896 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:01,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:01,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:01,896 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 19:54:01,896 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-08 19:54:01,896 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:01,897 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:01,897 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:01,898 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 19:54:01,898 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:01,898 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:01,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:01,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:01,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:01,898 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-08 19:54:01,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:01,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:01,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:01,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:01,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:01,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:01,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:01,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:01,902 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 19:54:01,902 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-08 19:54:01,902 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:01,902 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:01,902 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:01,902 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:01,904 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:01,904 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:01,904 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:01,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:01,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:01,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:01,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:01,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:01,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:01,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:01,905 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:01,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:01,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:01,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:01,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:01,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:01,906 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:01,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:01,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:01,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:01,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:01,909 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:01,909 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:01,909 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:01,909 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:01,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:01,909 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:01,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:01,909 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:01,909 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:01,909 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 19:54:01,909 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:01,909 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:01,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:01,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:01,909 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 19:54:01,909 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:01,909 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:01,910 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:01,910 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:02,010 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:02,012 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:02,017 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:02,019 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:02,019 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:02,019 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 19:54:02,019 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:02,020 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 19:54:02,020 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 19:54:02,021 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:02,021 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:02,023 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:02,023 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:02,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:02,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:02,024 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:02,024 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:02,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:02,025 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:02,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:02,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:02,026 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:02,026 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 19:54:02,026 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:02,026 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 19:54:02,026 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:02,027 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 19:54:02,027 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:02,027 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:02,027 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. 2023-03-08 19:54:02,027 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. started... 2023-03-08 19:54:02,027 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3551c80f1cc0fa597fb1a918e4b1b6f0 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:54:02,047 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228561a01a395ff45eabc217f980eee07c1 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228561a01a395ff45eabc217f980eee07c1 2023-03-08 19:54:02,054 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/.tmp/family1/010b66dd50be4ba193e98bb02b3bafc0 2023-03-08 19:54:02,059 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/.tmp/family1/010b66dd50be4ba193e98bb02b3bafc0 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/010b66dd50be4ba193e98bb02b3bafc0 2023-03-08 19:54:02,065 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/010b66dd50be4ba193e98bb02b3bafc0, entries=1, sequenceid=21, filesize=4.9 K 2023-03-08 19:54:02,066 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 3551c80f1cc0fa597fb1a918e4b1b6f0 in 39ms, sequenceid=21, compaction requested=true 2023-03-08 19:54:02,066 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3551c80f1cc0fa597fb1a918e4b1b6f0: 2023-03-08 19:54:02,066 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. 2023-03-08 19:54:02,066 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:02,066 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:02,066 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:02,066 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 19:54:02,066 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-08 19:54:02,068 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:02,068 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:02,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:02,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:02,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:02,069 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:02,069 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:02,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:02,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:02,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:02,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:02,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:02,070 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 19:54:02,070 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3c24d778[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:02,070 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 19:54:02,070 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:02,072 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:02,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:02,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:02,072 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 19:54:02,072 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-08 19:54:02,072 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:02,072 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:02,072 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:02,074 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 19:54:02,074 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:02,074 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:02,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:02,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:02,074 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-08 19:54:02,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:02,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:02,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:02,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:02,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:02,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:02,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:02,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:02,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:02,078 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 19:54:02,078 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-08 19:54:02,078 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:02,078 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:02,078 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:02,078 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:02,085 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:02,085 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:02,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:02,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:02,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:02,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:02,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:02,085 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:02,086 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:02,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:02,086 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:02,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:02,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:02,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:02,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:02,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:02,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:02,087 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:02,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:02,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:02,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:02,090 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:02,090 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:02,090 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:02,090 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:02,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:02,090 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:02,090 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:02,090 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 19:54:02,091 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:02,090 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:02,091 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:02,091 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:02,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:02,091 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:02,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:02,091 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:02,091 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:02,091 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:02,191 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:02,193 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:02,199 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:02,200 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:02,200 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:02,201 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 19:54:02,201 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:02,201 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 19:54:02,201 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 19:54:02,202 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:02,202 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:02,204 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:02,204 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:02,204 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:02,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:02,204 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:02,204 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:02,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:02,205 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:02,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:02,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:02,205 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:02,205 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 19:54:02,205 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:02,206 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 19:54:02,206 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:02,206 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 19:54:02,206 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:02,206 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:02,206 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. 2023-03-08 19:54:02,206 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. started... 2023-03-08 19:54:02,207 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3551c80f1cc0fa597fb1a918e4b1b6f0 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:54:02,624 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512313115778b718e43d485b68e064c2f6673 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512313115778b718e43d485b68e064c2f6673 2023-03-08 19:54:03,032 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/.tmp/family1/59ad4babcfad4f3c96192af8ab45b54b 2023-03-08 19:54:03,040 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/.tmp/family1/59ad4babcfad4f3c96192af8ab45b54b as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/59ad4babcfad4f3c96192af8ab45b54b 2023-03-08 19:54:03,046 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/59ad4babcfad4f3c96192af8ab45b54b, entries=1, sequenceid=25, filesize=4.9 K 2023-03-08 19:54:03,047 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 3551c80f1cc0fa597fb1a918e4b1b6f0 in 840ms, sequenceid=25, compaction requested=true 2023-03-08 19:54:03,047 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3551c80f1cc0fa597fb1a918e4b1b6f0: 2023-03-08 19:54:03,047 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. 2023-03-08 19:54:03,047 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:03,047 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:03,047 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:03,047 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 19:54:03,047 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-08 19:54:03,050 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:03,050 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:03,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:03,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:03,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:03,051 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:03,051 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:03,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:03,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:03,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:03,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:03,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:03,053 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 19:54:03,053 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4dc2627a[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:03,053 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 19:54:03,053 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:03,054 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:03,054 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:03,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:03,055 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 19:54:03,055 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-08 19:54:03,055 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:03,055 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:03,055 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:03,056 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:03,057 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 19:54:03,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:03,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:03,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:03,057 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:03,057 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-08 19:54:03,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:03,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:03,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:03,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:03,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:03,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:03,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:03,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:03,060 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 19:54:03,060 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-08 19:54:03,060 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:03,060 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:03,060 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:03,060 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:03,062 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:03,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:03,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:03,063 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:03,063 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:03,063 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:03,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:03,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:03,063 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:03,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:03,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:03,063 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:03,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:03,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:03,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:03,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:03,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:03,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:03,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:03,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:03,066 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:03,070 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:03,070 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:03,070 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:03,070 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:03,070 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 19:54:03,070 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:03,070 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:03,070 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:03,070 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:03,070 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:03,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:03,071 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:03,071 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:03,071 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:03,071 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:03,071 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:03,071 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:03,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:03,171 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:03,172 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:03,177 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:03,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:03,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:03,179 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 19:54:03,179 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:03,180 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 19:54:03,180 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 19:54:03,180 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:03,180 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:03,182 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:03,182 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:03,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:03,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:03,182 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:03,182 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:03,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:03,183 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:03,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:03,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:03,183 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:03,183 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 19:54:03,183 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:03,183 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 19:54:03,183 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:03,184 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 19:54:03,184 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:03,184 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:03,184 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. 2023-03-08 19:54:03,184 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. started... 2023-03-08 19:54:03,184 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3551c80f1cc0fa597fb1a918e4b1b6f0 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:54:03,601 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601016c4ae3bc0ffe47d9ae0fcd98164c4e55 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601016c4ae3bc0ffe47d9ae0fcd98164c4e55 2023-03-08 19:54:04,007 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/.tmp/family1/0d82a21ed1b9443a9cd808229ae4410f 2023-03-08 19:54:04,013 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/.tmp/family1/0d82a21ed1b9443a9cd808229ae4410f as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/0d82a21ed1b9443a9cd808229ae4410f 2023-03-08 19:54:04,018 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/0d82a21ed1b9443a9cd808229ae4410f, entries=1, sequenceid=29, filesize=4.9 K 2023-03-08 19:54:04,019 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 3551c80f1cc0fa597fb1a918e4b1b6f0 in 835ms, sequenceid=29, compaction requested=true 2023-03-08 19:54:04,019 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3551c80f1cc0fa597fb1a918e4b1b6f0: 2023-03-08 19:54:04,019 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. 2023-03-08 19:54:04,019 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:04,020 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:04,020 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:04,020 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 19:54:04,020 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-08 19:54:04,021 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,021 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:04,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:04,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:04,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:04,022 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,022 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:04,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:04,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:04,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:04,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:04,023 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 19:54:04,023 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7e445d69[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:04,023 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 19:54:04,023 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,024 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,025 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 19:54:04,025 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-08 19:54:04,025 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:04,025 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:04,025 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:04,027 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 19:54:04,027 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:04,027 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:04,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:04,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:04,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:04,027 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-08 19:54:04,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:04,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:04,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:04,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:04,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:04,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:04,030 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 19:54:04,030 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-08 19:54:04,030 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:04,030 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:04,030 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,030 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:04,032 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,032 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:04,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:04,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,032 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:04,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:04,033 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:04,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:04,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:04,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:04,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:04,034 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:04,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:04,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:04,036 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:04,036 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:04,036 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:04,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:04,036 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:04,036 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:04,036 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,037 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:04,037 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 19:54:04,037 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:04,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:04,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:04,037 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,038 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,038 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,038 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:04,038 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:04,138 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:04,139 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:04,145 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:04,147 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:04,147 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,147 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 19:54:04,147 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:04,148 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 19:54:04,148 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 19:54:04,148 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,148 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,150 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:04,150 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:04,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:04,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:04,150 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:04,150 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:04,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,151 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:04,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,151 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,151 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 19:54:04,151 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,151 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 19:54:04,151 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:04,152 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 19:54:04,152 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:04,152 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:04,152 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. 2023-03-08 19:54:04,152 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. started... 2023-03-08 19:54:04,152 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3551c80f1cc0fa597fb1a918e4b1b6f0 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:54:04,568 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103b0e499a4ced54a29b9baa397d59854ae to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103b0e499a4ced54a29b9baa397d59854ae 2023-03-08 19:54:04,974 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/.tmp/family1/3a846ebb6b1349c7b7a745336cfa1173 2023-03-08 19:54:04,980 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/.tmp/family1/3a846ebb6b1349c7b7a745336cfa1173 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/3a846ebb6b1349c7b7a745336cfa1173 2023-03-08 19:54:04,985 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/3a846ebb6b1349c7b7a745336cfa1173, entries=1, sequenceid=33, filesize=4.9 K 2023-03-08 19:54:04,986 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 3551c80f1cc0fa597fb1a918e4b1b6f0 in 834ms, sequenceid=33, compaction requested=true 2023-03-08 19:54:04,986 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3551c80f1cc0fa597fb1a918e4b1b6f0: 2023-03-08 19:54:04,986 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. 2023-03-08 19:54:04,986 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:04,986 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:04,986 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:04,986 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 19:54:04,986 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-08 19:54:04,988 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:04,988 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:04,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:04,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:04,989 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:04,989 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:04,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:04,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:04,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:04,990 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 19:54:04,990 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@427c1819[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:04,990 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 19:54:04,990 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,991 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,992 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 19:54:04,992 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-08 19:54:04,992 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:04,992 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:04,992 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:04,996 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 19:54:04,996 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:04,996 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:04,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:04,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:04,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:04,996 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-08 19:54:04,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:04,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:04,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:04,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:04,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:04,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:04,999 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 19:54:04,999 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-08 19:54:04,999 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:04,999 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:04,999 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:04,999 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:05,000 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,001 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,001 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:05,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:05,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:05,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:05,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:05,001 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:05,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:05,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:05,002 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:05,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:05,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:05,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:05,005 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:05,005 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:05,005 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:05,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:05,005 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,005 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 19:54:05,005 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:05,005 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 19:54:05,005 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:05,005 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:05,005 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,006 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:05,006 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:05,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:05,006 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:05,006 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,006 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:05,106 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:05,107 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:05,112 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:05,114 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:05,114 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,115 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 19:54:05,115 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:05,116 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 19:54:05,116 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 19:54:05,116 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,116 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,119 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:05,119 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:05,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:05,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:05,120 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:05,120 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:05,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,120 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:05,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,121 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,121 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 19:54:05,121 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,121 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 19:54:05,121 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:05,122 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 19:54:05,122 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:05,122 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:05,122 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. 2023-03-08 19:54:05,122 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. started... 2023-03-08 19:54:05,122 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3551c80f1cc0fa597fb1a918e4b1b6f0 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:54:05,538 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090719d5235c1727445eb83270b82c457f47 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015090719d5235c1727445eb83270b82c457f47 2023-03-08 19:54:05,944 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/.tmp/family1/8148948306a344f4b33574a099b332be 2023-03-08 19:54:05,950 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/.tmp/family1/8148948306a344f4b33574a099b332be as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/8148948306a344f4b33574a099b332be 2023-03-08 19:54:05,955 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/8148948306a344f4b33574a099b332be, entries=1, sequenceid=37, filesize=4.9 K 2023-03-08 19:54:05,957 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 3551c80f1cc0fa597fb1a918e4b1b6f0 in 835ms, sequenceid=37, compaction requested=true 2023-03-08 19:54:05,957 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3551c80f1cc0fa597fb1a918e4b1b6f0: 2023-03-08 19:54:05,957 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. 2023-03-08 19:54:05,957 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:05,957 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:05,957 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:05,957 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 19:54:05,957 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-08 19:54:05,959 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:05,959 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:05,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:05,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:05,959 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,959 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:05,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:05,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:05,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:05,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:05,960 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 19:54:05,961 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@58f99839[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:05,961 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 19:54:05,961 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,962 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,962 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 19:54:05,962 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-08 19:54:05,962 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:05,962 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:05,962 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:05,964 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:05,964 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 19:54:05,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:05,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:05,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:05,964 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:05,964 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-08 19:54:05,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:05,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:05,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:05,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:05,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:05,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:05,967 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 19:54:05,967 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-08 19:54:05,967 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:05,967 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:05,967 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,967 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:05,970 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,970 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:05,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,970 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:05,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:05,970 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:05,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:05,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:05,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:05,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:05,972 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:05,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:05,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:05,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,974 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:05,974 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:05,974 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,974 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 19:54:05,974 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:05,974 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:05,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:05,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:05,974 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:05,975 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 19:54:05,974 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,975 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:05,975 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:05,975 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:05,975 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:05,976 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:05,976 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:06,076 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:06,077 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:06,082 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:06,084 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:06,084 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:06,084 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 19:54:06,084 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:06,085 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 19:54:06,085 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 19:54:06,085 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:06,085 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:06,086 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:06,086 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:06,086 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:06,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:06,087 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:06,087 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:06,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:06,087 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:06,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:06,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:06,088 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:06,088 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 19:54:06,088 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:06,088 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 19:54:06,088 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:06,089 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 19:54:06,089 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:06,089 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:06,089 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. 2023-03-08 19:54:06,090 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. started... 2023-03-08 19:54:06,090 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3551c80f1cc0fa597fb1a918e4b1b6f0 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:54:06,109 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120457dfc1f01904d839d1118704f04d68c to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120457dfc1f01904d839d1118704f04d68c 2023-03-08 19:54:06,518 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/.tmp/family1/a5ce9f6232b24783884361cae84b1865 2023-03-08 19:54:06,524 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/.tmp/family1/a5ce9f6232b24783884361cae84b1865 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/a5ce9f6232b24783884361cae84b1865 2023-03-08 19:54:06,530 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/a5ce9f6232b24783884361cae84b1865, entries=1, sequenceid=41, filesize=4.9 K 2023-03-08 19:54:06,531 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 3551c80f1cc0fa597fb1a918e4b1b6f0 in 441ms, sequenceid=41, compaction requested=true 2023-03-08 19:54:06,531 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3551c80f1cc0fa597fb1a918e4b1b6f0: 2023-03-08 19:54:06,531 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. 2023-03-08 19:54:06,531 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:06,531 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:06,531 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:06,531 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 19:54:06,531 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-08 19:54:06,533 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:06,533 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:06,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:06,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:06,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:06,533 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:06,533 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:06,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:06,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:06,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:06,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:06,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:06,534 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 19:54:06,534 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@676f0ee[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:06,534 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 19:54:06,535 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:06,537 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:06,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:06,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:06,537 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 19:54:06,537 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-08 19:54:06,537 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:06,537 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:06,537 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:06,539 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:06,539 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 19:54:06,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:06,539 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:06,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:06,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:06,539 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-08 19:54:06,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:06,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:06,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:06,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:06,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:06,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:06,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:06,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:06,542 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 19:54:06,542 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-08 19:54:06,542 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:06,542 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:06,542 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:06,542 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:06,543 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:06,543 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:06,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:06,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:06,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:06,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:06,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:06,543 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:06,544 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:06,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:06,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:06,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:06,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:06,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:06,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:06,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:06,545 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:06,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:06,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:06,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:06,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:06,547 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:06,547 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:06,547 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:06,547 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:06,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:06,547 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:06,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:06,547 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:06,547 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 19:54:06,547 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-08 19:54:06,549 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:06,549 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:06,549 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:06,549 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:06,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:06,549 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:06,549 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:06,549 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:06,549 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:06,649 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:06,651 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:06,652 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39989] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithMonthlyPolicy 2023-03-08 19:54:06,654 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] procedure2.ProcedureExecutor(1029): Stored pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 19:54:06,654 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 19:54:06,655 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 19:54:06,655 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=22, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678305846655 2023-03-08 19:54:06,656 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 19:54:06,658 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithMonthlyPolicy' and column 'family1' 2023-03-08 19:54:06,658 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 19:54:06,658 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] zookeeper.ReadOnlyZKClient(139): Connect 0x1b538d9e to 127.0.0.1:56751 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 19:54:06,664 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3458c5b3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 19:54:06,664 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 19:54:07,499 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601033615555e3afc42da9fdd6a24e15c03c5 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601033615555e3afc42da9fdd6a24e15c03c5 2023-03-08 19:54:07,500 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 19:54:07,514 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 19:54:07,515 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40922, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 19:54:07,522 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 19:54:07,523 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38630, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 19:54:07,537 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/d1c7e6023c834a5297c7700368e972fb first=Optional[r5] last=Optional[r6] 2023-03-08 19:54:07,537 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0., hostname=jenkins-hbase4.apache.org,33367,1678305218680, seqNum=2 for row with hfile group [{family1,hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/d1c7e6023c834a5297c7700368e972fb}] 2023-03-08 19:54:07,542 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(763): Validating hfile at hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/d1c7e6023c834a5297c7700368e972fb for inclusion in 3551c80f1cc0fa597fb1a918e4b1b6f0/family1 2023-03-08 19:54:07,546 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-03-08 19:54:07,546 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 19:54:07,546 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(2446): Flush status journal for 3551c80f1cc0fa597fb1a918e4b1b6f0: 2023-03-08 19:54:07,547 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/d1c7e6023c834a5297c7700368e972fb to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithMonthlyPolicy__j7tqloogo7ehve9lntqq5vpb4bj4mjpl36cbjae0adar3ap9onk0o5t9c2iub0ho/family1/d1c7e6023c834a5297c7700368e972fb 2023-03-08 19:54:07,549 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithMonthlyPolicy__j7tqloogo7ehve9lntqq5vpb4bj4mjpl36cbjae0adar3ap9onk0o5t9c2iub0ho/family1/d1c7e6023c834a5297c7700368e972fb as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/322dfd001cc54e49ad686a51cb665cea_SeqId_44_ 2023-03-08 19:54:07,550 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(849): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithMonthlyPolicy__j7tqloogo7ehve9lntqq5vpb4bj4mjpl36cbjae0adar3ap9onk0o5t9c2iub0ho/family1/d1c7e6023c834a5297c7700368e972fb into 3551c80f1cc0fa597fb1a918e4b1b6f0/family1 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/322dfd001cc54e49ad686a51cb665cea_SeqId_44_ - updating store file list. 2023-03-08 19:54:07,555 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(882): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/322dfd001cc54e49ad686a51cb665cea_SeqId_44_ into 3551c80f1cc0fa597fb1a918e4b1b6f0/family1 2023-03-08 19:54:07,555 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(855): Successfully loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithMonthlyPolicy__j7tqloogo7ehve9lntqq5vpb4bj4mjpl36cbjae0adar3ap9onk0o5t9c2iub0ho/family1/d1c7e6023c834a5297c7700368e972fb into 3551c80f1cc0fa597fb1a918e4b1b6f0/family1 (new location: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/322dfd001cc54e49ad686a51cb665cea_SeqId_44_) 2023-03-08 19:54:07,555 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithMonthlyPolicy__j7tqloogo7ehve9lntqq5vpb4bj4mjpl36cbjae0adar3ap9onk0o5t9c2iub0ho/family1/d1c7e6023c834a5297c7700368e972fb 2023-03-08 19:54:07,556 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-08 19:54:07,559 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-08 19:54:07,560 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(1912): 3551c80f1cc0fa597fb1a918e4b1b6f0/family1 is initiating minor compaction 2023-03-08 19:54:07,560 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@728e2713; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 19:54:07,560 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 3551c80f1cc0fa597fb1a918e4b1b6f0/family1 in testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. 2023-03-08 19:54:07,560 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/73e2073e030d46a78276019b6f4cdda4, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/c0175e335d6a465393aa5af881be82e3, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/da978af043f947a9abe4c8afd05ed840, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/010b66dd50be4ba193e98bb02b3bafc0, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/59ad4babcfad4f3c96192af8ab45b54b, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/0d82a21ed1b9443a9cd808229ae4410f, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/3a846ebb6b1349c7b7a745336cfa1173, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/8148948306a344f4b33574a099b332be, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/a5ce9f6232b24783884361cae84b1865, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/322dfd001cc54e49ad686a51cb665cea_SeqId_44_] into tmpdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/.tmp, totalSize=49.7 K 2023-03-08 19:54:07,560 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 73e2073e030d46a78276019b6f4cdda4, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-08 19:54:07,560 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting c0175e335d6a465393aa5af881be82e3, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-08 19:54:07,560 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting da978af043f947a9abe4c8afd05ed840, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-08 19:54:07,560 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 010b66dd50be4ba193e98bb02b3bafc0, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-08 19:54:07,560 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 59ad4babcfad4f3c96192af8ab45b54b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-08 19:54:07,560 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 0d82a21ed1b9443a9cd808229ae4410f, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-08 19:54:07,560 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 3a846ebb6b1349c7b7a745336cfa1173, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-08 19:54:07,561 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 8148948306a344f4b33574a099b332be, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-08 19:54:07,561 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting a5ce9f6232b24783884361cae84b1865, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-08 19:54:07,561 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 322dfd001cc54e49ad686a51cb665cea_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-08 19:54:07,562 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 19:54:07,563 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:54:07,565 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601016c4ae3bc0ffe47d9ae0fcd98164c4e55 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601016c4ae3bc0ffe47d9ae0fcd98164c4e55 2023-03-08 19:54:07,566 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103b0e499a4ced54a29b9baa397d59854ae to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103b0e499a4ced54a29b9baa397d59854ae 2023-03-08 19:54:07,566 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-03-08 19:54:07,566 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-03-08 19:54:07,567 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 19:54:07,567 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 19:54:07,606 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] throttle.PressureAwareThroughputController(145): 3551c80f1cc0fa597fb1a918e4b1b6f0#family1#compaction#22 average throughput is 0.68 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 19:54:07,824 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 19:54:07,824 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x72d632fe to 127.0.0.1:56751 2023-03-08 19:54:07,824 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 19:54:07,825 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 19:54:07,826 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x11e88228 to 127.0.0.1:56751 2023-03-08 19:54:07,826 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 19:54:08,025 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/.tmp/family1/67c0c17ed8e245abb0838416a0dcffce as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/67c0c17ed8e245abb0838416a0dcffce 2023-03-08 19:54:08,025 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231583bb3b1a71d44959df7a13a0374df6b to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231583bb3b1a71d44959df7a13a0374df6b 2023-03-08 19:54:08,026 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 19:54:08,032 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 3551c80f1cc0fa597fb1a918e4b1b6f0/family1 of 3551c80f1cc0fa597fb1a918e4b1b6f0 into 67c0c17ed8e245abb0838416a0dcffce(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 19:54:08,032 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 3551c80f1cc0fa597fb1a918e4b1b6f0: 2023-03-08 19:54:08,032 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1678305247559; duration=0sec 2023-03-08 19:54:08,033 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 19:54:08,054 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/8c47e6e25ef84b4397fa419bb7994a22 first=Optional[r1] last=Optional[r4] 2023-03-08 19:54:08,055 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0., hostname=jenkins-hbase4.apache.org,33367,1678305218680, seqNum=2 for row with hfile group [{family1,hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/8c47e6e25ef84b4397fa419bb7994a22}] 2023-03-08 19:54:08,057 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(763): Validating hfile at hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/8c47e6e25ef84b4397fa419bb7994a22 for inclusion in 3551c80f1cc0fa597fb1a918e4b1b6f0/family1 2023-03-08 19:54:08,060 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-03-08 19:54:08,060 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 19:54:08,060 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(2446): Flush status journal for 3551c80f1cc0fa597fb1a918e4b1b6f0: 2023-03-08 19:54:08,061 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/8c47e6e25ef84b4397fa419bb7994a22 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithMonthlyPolicy__4ejd9efvasbs021ft4udq9j622p77fchgj4qrd1avu18apvirejsttm6rbeo4tei/family1/8c47e6e25ef84b4397fa419bb7994a22 2023-03-08 19:54:08,063 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithMonthlyPolicy__4ejd9efvasbs021ft4udq9j622p77fchgj4qrd1avu18apvirejsttm6rbeo4tei/family1/8c47e6e25ef84b4397fa419bb7994a22 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/b0dbc207443449bda946b5c75c0a2305_SeqId_47_ 2023-03-08 19:54:08,064 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(849): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithMonthlyPolicy__4ejd9efvasbs021ft4udq9j622p77fchgj4qrd1avu18apvirejsttm6rbeo4tei/family1/8c47e6e25ef84b4397fa419bb7994a22 into 3551c80f1cc0fa597fb1a918e4b1b6f0/family1 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/b0dbc207443449bda946b5c75c0a2305_SeqId_47_ - updating store file list. 2023-03-08 19:54:08,068 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(882): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/b0dbc207443449bda946b5c75c0a2305_SeqId_47_ into 3551c80f1cc0fa597fb1a918e4b1b6f0/family1 2023-03-08 19:54:08,068 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(855): Successfully loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithMonthlyPolicy__4ejd9efvasbs021ft4udq9j622p77fchgj4qrd1avu18apvirejsttm6rbeo4tei/family1/8c47e6e25ef84b4397fa419bb7994a22 into 3551c80f1cc0fa597fb1a918e4b1b6f0/family1 (new location: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/b0dbc207443449bda946b5c75c0a2305_SeqId_47_) 2023-03-08 19:54:08,069 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithMonthlyPolicy__4ejd9efvasbs021ft4udq9j622p77fchgj4qrd1avu18apvirejsttm6rbeo4tei/family1/8c47e6e25ef84b4397fa419bb7994a22 2023-03-08 19:54:08,069 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-08 19:54:08,070 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-08 19:54:08,070 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(1912): 3551c80f1cc0fa597fb1a918e4b1b6f0/family1 is initiating minor compaction (all files) 2023-03-08 19:54:08,070 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@3a6d3029; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 19:54:08,070 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 3551c80f1cc0fa597fb1a918e4b1b6f0/family1 in testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. 2023-03-08 19:54:08,071 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/4be98f24a4ef4eb6803e1ce815aa33e8, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/67c0c17ed8e245abb0838416a0dcffce, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/b0dbc207443449bda946b5c75c0a2305_SeqId_47_] into tmpdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/.tmp, totalSize=178.1 K 2023-03-08 19:54:08,071 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 4be98f24a4ef4eb6803e1ce815aa33e8, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878843488 2023-03-08 19:54:08,072 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 67c0c17ed8e245abb0838416a0dcffce, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621243488 2023-03-08 19:54:08,072 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting b0dbc207443449bda946b5c75c0a2305_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965243488 2023-03-08 19:54:08,072 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 19:54:08,073 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:54:08,075 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120176e8ffa363a141bdb010303788de0801 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120176e8ffa363a141bdb010303788de0801 2023-03-08 19:54:08,076 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205ae2385370f3e4ed683a6dc90374af25c to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205ae2385370f3e4ed683a6dc90374af25c 2023-03-08 19:54:08,078 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228561a01a395ff45eabc217f980eee07c1 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228561a01a395ff45eabc217f980eee07c1 2023-03-08 19:54:08,079 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512313115778b718e43d485b68e064c2f6673 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512313115778b718e43d485b68e064c2f6673 2023-03-08 19:54:08,079 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-03-08 19:54:08,079 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-03-08 19:54:08,080 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 19:54:08,080 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 19:54:08,089 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] throttle.PressureAwareThroughputController(145): 3551c80f1cc0fa597fb1a918e4b1b6f0#family1#compaction#23 average throughput is 19.45 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 19:54:08,122 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/73e2073e030d46a78276019b6f4cdda4, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/c0175e335d6a465393aa5af881be82e3, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/da978af043f947a9abe4c8afd05ed840, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/010b66dd50be4ba193e98bb02b3bafc0, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/59ad4babcfad4f3c96192af8ab45b54b, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/0d82a21ed1b9443a9cd808229ae4410f, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/3a846ebb6b1349c7b7a745336cfa1173, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/8148948306a344f4b33574a099b332be, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/a5ce9f6232b24783884361cae84b1865, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/322dfd001cc54e49ad686a51cb665cea_SeqId_44_] to archive 2023-03-08 19:54:08,123 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:54:08,127 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/73e2073e030d46a78276019b6f4cdda4 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/73e2073e030d46a78276019b6f4cdda4 2023-03-08 19:54:08,129 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/c0175e335d6a465393aa5af881be82e3 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/c0175e335d6a465393aa5af881be82e3 2023-03-08 19:54:08,132 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/da978af043f947a9abe4c8afd05ed840 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/da978af043f947a9abe4c8afd05ed840 2023-03-08 19:54:08,132 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/.tmp/family1/7fa5f91a7ef946fb99d736d789961b28 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/7fa5f91a7ef946fb99d736d789961b28 2023-03-08 19:54:08,135 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/010b66dd50be4ba193e98bb02b3bafc0 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/010b66dd50be4ba193e98bb02b3bafc0 2023-03-08 19:54:08,137 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/59ad4babcfad4f3c96192af8ab45b54b to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/59ad4babcfad4f3c96192af8ab45b54b 2023-03-08 19:54:08,138 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/0d82a21ed1b9443a9cd808229ae4410f to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/0d82a21ed1b9443a9cd808229ae4410f 2023-03-08 19:54:08,140 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/3a846ebb6b1349c7b7a745336cfa1173 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/3a846ebb6b1349c7b7a745336cfa1173 2023-03-08 19:54:08,142 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/8148948306a344f4b33574a099b332be to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/8148948306a344f4b33574a099b332be 2023-03-08 19:54:08,144 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/a5ce9f6232b24783884361cae84b1865 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/a5ce9f6232b24783884361cae84b1865 2023-03-08 19:54:08,144 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 3551c80f1cc0fa597fb1a918e4b1b6f0/family1 of 3551c80f1cc0fa597fb1a918e4b1b6f0 into 7fa5f91a7ef946fb99d736d789961b28(size=167.7 K), total size for store is 167.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 19:54:08,144 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 3551c80f1cc0fa597fb1a918e4b1b6f0: 2023-03-08 19:54:08,145 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1678305248070; duration=0sec 2023-03-08 19:54:08,145 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 19:54:08,146 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/322dfd001cc54e49ad686a51cb665cea_SeqId_44_ to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/322dfd001cc54e49ad686a51cb665cea_SeqId_44_ 2023-03-08 19:54:08,210 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/4be98f24a4ef4eb6803e1ce815aa33e8, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/67c0c17ed8e245abb0838416a0dcffce, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/b0dbc207443449bda946b5c75c0a2305_SeqId_47_] to archive 2023-03-08 19:54:08,211 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:54:08,213 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/4be98f24a4ef4eb6803e1ce815aa33e8 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/4be98f24a4ef4eb6803e1ce815aa33e8 2023-03-08 19:54:08,215 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/67c0c17ed8e245abb0838416a0dcffce to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/67c0c17ed8e245abb0838416a0dcffce 2023-03-08 19:54:08,216 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/b0dbc207443449bda946b5c75c0a2305_SeqId_47_ to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/b0dbc207443449bda946b5c75c0a2305_SeqId_47_ 2023-03-08 19:54:08,554 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130c8d77e078805478d9816616d1b296969 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130c8d77e078805478d9816616d1b296969 2023-03-08 19:54:08,555 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 19:54:08,580 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/faf09305f9b44d078386c3c7afe4f750 first=Optional[r00] last=Optional[r8] 2023-03-08 19:54:08,581 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0., hostname=jenkins-hbase4.apache.org,33367,1678305218680, seqNum=2 for row with hfile group [{family1,hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/faf09305f9b44d078386c3c7afe4f750}] 2023-03-08 19:54:08,583 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(763): Validating hfile at hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/faf09305f9b44d078386c3c7afe4f750 for inclusion in 3551c80f1cc0fa597fb1a918e4b1b6f0/family1 2023-03-08 19:54:08,586 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-03-08 19:54:08,586 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 19:54:08,586 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(2446): Flush status journal for 3551c80f1cc0fa597fb1a918e4b1b6f0: 2023-03-08 19:54:08,587 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/faf09305f9b44d078386c3c7afe4f750 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithMonthlyPolicy__39ri2sl7jlpq94i2l0mngdcpfp29f3va7lg5shmr44c0ufqt95e3452f7aq18kbe/family1/faf09305f9b44d078386c3c7afe4f750 2023-03-08 19:54:08,589 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithMonthlyPolicy__39ri2sl7jlpq94i2l0mngdcpfp29f3va7lg5shmr44c0ufqt95e3452f7aq18kbe/family1/faf09305f9b44d078386c3c7afe4f750 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/ab6918a20f574478ad088358cffb869f_SeqId_50_ 2023-03-08 19:54:08,589 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(849): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithMonthlyPolicy__39ri2sl7jlpq94i2l0mngdcpfp29f3va7lg5shmr44c0ufqt95e3452f7aq18kbe/family1/faf09305f9b44d078386c3c7afe4f750 into 3551c80f1cc0fa597fb1a918e4b1b6f0/family1 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/ab6918a20f574478ad088358cffb869f_SeqId_50_ - updating store file list. 2023-03-08 19:54:08,593 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(882): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/ab6918a20f574478ad088358cffb869f_SeqId_50_ into 3551c80f1cc0fa597fb1a918e4b1b6f0/family1 2023-03-08 19:54:08,593 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(855): Successfully loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithMonthlyPolicy__39ri2sl7jlpq94i2l0mngdcpfp29f3va7lg5shmr44c0ufqt95e3452f7aq18kbe/family1/faf09305f9b44d078386c3c7afe4f750 into 3551c80f1cc0fa597fb1a918e4b1b6f0/family1 (new location: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/family1/ab6918a20f574478ad088358cffb869f_SeqId_50_) 2023-03-08 19:54:08,594 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithMonthlyPolicy__39ri2sl7jlpq94i2l0mngdcpfp29f3va7lg5shmr44c0ufqt95e3452f7aq18kbe/family1/faf09305f9b44d078386c3c7afe4f750 2023-03-08 19:54:08,596 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 19:54:08,596 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:54:08,598 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120457dfc1f01904d839d1118704f04d68c to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120457dfc1f01904d839d1118704f04d68c 2023-03-08 19:54:08,599 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511282219db87b90a443e82956787b41d4d4e to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511282219db87b90a443e82956787b41d4d4e 2023-03-08 19:54:08,600 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511300a95ab5edc234e8f8453cfb8c755eb23 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511300a95ab5edc234e8f8453cfb8c755eb23 2023-03-08 19:54:08,600 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-03-08 19:54:08,600 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-08 19:54:08,601 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 19:54:08,601 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 19:54:08,601 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 19:54:08,601 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 19:54:08,601 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 19:54:08,602 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=22, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 19:54:08,604 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=22, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 1.9490 sec 2023-03-08 19:54:08,702 INFO [Listener at localhost/34049] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=250 (was 248) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-305148072_15 at /127.0.0.1:40428 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/36526ac9-e9a0-8ea5-8063-2f26c591d24b/cluster_63dc3982-03ab-365c-777c-42eb5a69ef67/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-305148072_15 at /127.0.0.1:40346 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-305148072_15 at /127.0.0.1:40230 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-305148072_15 at /127.0.0.1:40406 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-305148072_15 at /127.0.0.1:40268 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x66bb1497-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x66bb1497-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1ac1bbe3-shared-pool-23 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-305148072_15 at /127.0.0.1:40126 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-305148072_15 at /127.0.0.1:40314 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-305148072_15 at /127.0.0.1:40282 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-305148072_15 at /127.0.0.1:40342 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x66bb1497-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56751@0x1b538d9e-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: hconnection-0x66bb1497-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/36526ac9-e9a0-8ea5-8063-2f26c591d24b/cluster_63dc3982-03ab-365c-777c-42eb5a69ef67/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-305148072_15 at /127.0.0.1:40208 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56751@0x1b538d9e-SendThread(127.0.0.1:56751) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56751@0x1b538d9e sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/512641589.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x66bb1497-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-305148072_15 at /127.0.0.1:40246 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-305148072_15 at /127.0.0.1:40176 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x66bb1497-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-305148072_15 at /127.0.0.1:40262 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-305148072_15 at /127.0.0.1:40200 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-305148072_15 at /127.0.0.1:40298 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_968202085_15 at /127.0.0.1:40308 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_968202085_15 at /127.0.0.1:40158 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x66bb1497-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=595 (was 598), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=250 (was 295), ProcessCount=170 (was 170), AvailableMemoryMB=8826 (was 8238) - AvailableMemoryMB LEAK? - 2023-03-08 19:54:08,714 INFO [Listener at localhost/34049] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=250, OpenFileDescriptor=595, MaxFileDescriptor=60000, SystemLoadAverage=250, ProcessCount=170, AvailableMemoryMB=8826 2023-03-08 19:54:08,716 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-08 19:54:08,718 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure2.ProcedureExecutor(1029): Stored pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:08,719 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 19:54:08,719 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithWeeklyPolicy" procId is: 23 2023-03-08 19:54:08,720 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 19:54:08,720 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-08 19:54:08,722 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6 2023-03-08 19:54:08,722 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6 empty. 2023-03-08 19:54:08,723 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6 2023-03-08 19:54:08,723 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithWeeklyPolicy regions 2023-03-08 19:54:08,731 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-08 19:54:08,733 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => c0d4af7836e0c3ca2f373a671b92c8f6, NAME => 'testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp 2023-03-08 19:54:08,821 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-08 19:54:09,023 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-08 19:54:09,140 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:54:09,140 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1604): Closing c0d4af7836e0c3ca2f373a671b92c8f6, disabling compactions & flushes 2023-03-08 19:54:09,140 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. 2023-03-08 19:54:09,140 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. 2023-03-08 19:54:09,140 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. after waiting 0 ms 2023-03-08 19:54:09,140 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. 2023-03-08 19:54:09,140 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1838): Closed testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. 2023-03-08 19:54:09,140 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for c0d4af7836e0c3ca2f373a671b92c8f6: 2023-03-08 19:54:09,143 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 19:54:09,144 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678305249143"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678305249143"}]},"ts":"1678305249143"} 2023-03-08 19:54:09,145 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-08 19:54:09,146 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 19:54:09,146 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678305249146"}]},"ts":"1678305249146"} 2023-03-08 19:54:09,147 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-03-08 19:54:09,154 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=c0d4af7836e0c3ca2f373a671b92c8f6, ASSIGN}] 2023-03-08 19:54:09,155 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=c0d4af7836e0c3ca2f373a671b92c8f6, ASSIGN 2023-03-08 19:54:09,156 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=c0d4af7836e0c3ca2f373a671b92c8f6, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33367,1678305218680; forceNewPlan=false, retain=false 2023-03-08 19:54:09,307 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=c0d4af7836e0c3ca2f373a671b92c8f6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:09,307 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678305249307"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678305249307"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678305249307"}]},"ts":"1678305249307"} 2023-03-08 19:54:09,309 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=24, state=RUNNABLE; OpenRegionProcedure c0d4af7836e0c3ca2f373a671b92c8f6, server=jenkins-hbase4.apache.org,33367,1678305218680}] 2023-03-08 19:54:09,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-08 19:54:09,465 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. 2023-03-08 19:54:09,465 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c0d4af7836e0c3ca2f373a671b92c8f6, NAME => 'testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6.', STARTKEY => '', ENDKEY => ''} 2023-03-08 19:54:09,465 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithWeeklyPolicy c0d4af7836e0c3ca2f373a671b92c8f6 2023-03-08 19:54:09,465 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:54:09,465 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for c0d4af7836e0c3ca2f373a671b92c8f6 2023-03-08 19:54:09,465 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for c0d4af7836e0c3ca2f373a671b92c8f6 2023-03-08 19:54:09,466 INFO [StoreOpener-c0d4af7836e0c3ca2f373a671b92c8f6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region c0d4af7836e0c3ca2f373a671b92c8f6 2023-03-08 19:54:09,468 DEBUG [StoreOpener-c0d4af7836e0c3ca2f373a671b92c8f6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1 2023-03-08 19:54:09,468 DEBUG [StoreOpener-c0d4af7836e0c3ca2f373a671b92c8f6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1 2023-03-08 19:54:09,468 INFO [StoreOpener-c0d4af7836e0c3ca2f373a671b92c8f6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c0d4af7836e0c3ca2f373a671b92c8f6 columnFamilyName family1 2023-03-08 19:54:09,470 INFO [StoreOpener-c0d4af7836e0c3ca2f373a671b92c8f6-1] regionserver.HStore(310): Store=c0d4af7836e0c3ca2f373a671b92c8f6/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:54:09,471 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6 2023-03-08 19:54:09,471 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6 2023-03-08 19:54:09,473 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for c0d4af7836e0c3ca2f373a671b92c8f6 2023-03-08 19:54:09,476 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 19:54:09,476 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened c0d4af7836e0c3ca2f373a671b92c8f6; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74830743, jitterRate=0.11506496369838715}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 19:54:09,476 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for c0d4af7836e0c3ca2f373a671b92c8f6: 2023-03-08 19:54:09,477 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6., pid=25, masterSystemTime=1678305249461 2023-03-08 19:54:09,479 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. 2023-03-08 19:54:09,480 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. 2023-03-08 19:54:09,480 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=c0d4af7836e0c3ca2f373a671b92c8f6, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:09,480 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678305249480"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678305249480"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678305249480"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678305249480"}]},"ts":"1678305249480"} 2023-03-08 19:54:09,483 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=24 2023-03-08 19:54:09,484 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=24, state=SUCCESS; OpenRegionProcedure c0d4af7836e0c3ca2f373a671b92c8f6, server=jenkins-hbase4.apache.org,33367,1678305218680 in 173 msec 2023-03-08 19:54:09,485 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=23 2023-03-08 19:54:09,485 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=23, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=c0d4af7836e0c3ca2f373a671b92c8f6, ASSIGN in 330 msec 2023-03-08 19:54:09,486 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 19:54:09,486 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678305249486"}]},"ts":"1678305249486"} 2023-03-08 19:54:09,488 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-03-08 19:54:09,491 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 19:54:09,493 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=23, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy in 774 msec 2023-03-08 19:54:09,825 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-08 19:54:09,825 INFO [Listener at localhost/34049] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithWeeklyPolicy, procId: 23 completed 2023-03-08 19:54:09,851 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:09,853 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:09,853 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 19:54:09,854 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:09,854 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 19:54:09,854 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 19:54:09,854 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:09,854 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:09,856 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:09,856 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:09,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:09,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:09,856 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:09,856 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:09,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:09,857 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:09,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:09,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:09,857 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:09,857 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:09,857 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 19:54:09,857 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:09,857 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 19:54:09,858 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:09,858 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:09,858 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. 2023-03-08 19:54:09,858 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. started... 2023-03-08 19:54:09,859 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c0d4af7836e0c3ca2f373a671b92c8f6 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-08 19:54:10,280 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113086483e2251964fd6ab54b0e2e28ec714 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015113086483e2251964fd6ab54b0e2e28ec714 2023-03-08 19:54:10,684 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/.tmp/family1/943c65840cfa44a8932da0ff713746be 2023-03-08 19:54:10,690 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/.tmp/family1/943c65840cfa44a8932da0ff713746be as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/943c65840cfa44a8932da0ff713746be 2023-03-08 19:54:10,695 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/943c65840cfa44a8932da0ff713746be, entries=1000, sequenceid=5, filesize=165.1 K 2023-03-08 19:54:10,696 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for c0d4af7836e0c3ca2f373a671b92c8f6 in 838ms, sequenceid=5, compaction requested=false 2023-03-08 19:54:10,696 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 19:54:10,697 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c0d4af7836e0c3ca2f373a671b92c8f6: 2023-03-08 19:54:10,697 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. 2023-03-08 19:54:10,697 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:10,697 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:10,697 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:10,697 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 19:54:10,697 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-08 19:54:10,699 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:10,699 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:10,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:10,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:10,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:10,699 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:10,699 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:10,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:10,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:10,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:10,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:10,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:10,700 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 19:54:10,700 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5a054a5c[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:10,700 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 19:54:10,701 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:10,703 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:10,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:10,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:10,703 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 19:54:10,703 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-08 19:54:10,703 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:10,703 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:10,703 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:10,705 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:10,705 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 19:54:10,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:10,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:10,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:10,706 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:10,706 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-08 19:54:10,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:10,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:10,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:10,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:10,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:10,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:10,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:10,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:10,708 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 19:54:10,708 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-08 19:54:10,708 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:10,708 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:10,708 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:10,708 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:10,711 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:10,711 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:10,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:10,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:10,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:10,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:10,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:10,711 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:10,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:10,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:10,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:10,711 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:10,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:10,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:10,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:10,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:10,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:10,712 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:10,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:10,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:10,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:10,716 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:10,716 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:10,716 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:10,716 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:10,716 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:10,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:10,716 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:10,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:10,716 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:10,716 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 19:54:10,716 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:10,716 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:10,716 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:10,716 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:10,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:10,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:10,717 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:10,717 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:10,817 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:10,818 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:10,822 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:10,824 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:10,824 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:10,824 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 19:54:10,824 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:10,824 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 19:54:10,824 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 19:54:10,824 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:10,825 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:10,826 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:10,826 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:10,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:10,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:10,827 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:10,827 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:10,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:10,827 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:10,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:10,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:10,828 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:10,828 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 19:54:10,828 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:10,828 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 19:54:10,828 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:10,828 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 19:54:10,828 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:10,828 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:10,828 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. 2023-03-08 19:54:10,828 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. started... 2023-03-08 19:54:10,828 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c0d4af7836e0c3ca2f373a671b92c8f6 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-08 19:54:11,242 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128e46513bee0e544f49ac7af2cf7fceff7 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151128e46513bee0e544f49ac7af2cf7fceff7 2023-03-08 19:54:11,649 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/.tmp/family1/88e43825322d446a9f8465cc968de562 2023-03-08 19:54:11,654 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/.tmp/family1/88e43825322d446a9f8465cc968de562 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/88e43825322d446a9f8465cc968de562 2023-03-08 19:54:11,659 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/88e43825322d446a9f8465cc968de562, entries=1, sequenceid=9, filesize=5.0 K 2023-03-08 19:54:11,660 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for c0d4af7836e0c3ca2f373a671b92c8f6 in 832ms, sequenceid=9, compaction requested=false 2023-03-08 19:54:11,660 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c0d4af7836e0c3ca2f373a671b92c8f6: 2023-03-08 19:54:11,660 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. 2023-03-08 19:54:11,660 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:11,660 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:11,660 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:11,661 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 19:54:11,661 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-08 19:54:11,662 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:11,662 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:11,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:11,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:11,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:11,663 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:11,663 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:11,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:11,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:11,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:11,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:11,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:11,664 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 19:54:11,664 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@781b3dd5[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:11,664 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 19:54:11,664 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:11,665 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:11,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:11,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:11,665 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 19:54:11,666 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-08 19:54:11,666 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:11,666 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:11,666 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:11,667 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:11,667 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 19:54:11,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:11,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:11,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:11,667 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:11,667 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-08 19:54:11,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:11,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:11,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:11,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:11,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:11,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:11,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:11,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:11,669 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 19:54:11,669 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-08 19:54:11,669 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:11,669 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:11,669 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:11,669 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:11,672 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:11,672 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:11,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:11,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:11,672 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:11,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:11,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:11,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:11,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:11,672 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:11,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:11,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:11,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:11,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:11,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:11,673 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:11,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:11,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:11,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:11,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:11,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:11,676 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:11,676 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:11,676 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:11,676 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:11,676 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:11,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:11,676 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 19:54:11,676 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:11,676 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 19:54:11,676 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:11,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:11,676 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:11,676 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:11,676 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:11,676 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:11,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:11,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:11,676 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:11,677 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:11,776 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:11,777 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:11,782 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:11,783 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:11,784 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:11,784 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 19:54:11,784 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:11,784 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 19:54:11,784 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 19:54:11,784 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:11,784 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:11,786 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:11,786 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:11,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:11,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:11,786 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:11,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:11,786 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:11,786 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:11,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:11,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:11,787 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:11,787 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 19:54:11,787 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:11,787 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 19:54:11,787 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:11,787 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 19:54:11,787 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:11,787 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:11,787 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. 2023-03-08 19:54:11,788 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. started... 2023-03-08 19:54:11,788 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c0d4af7836e0c3ca2f373a671b92c8f6 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:54:12,200 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512015d2b5914fd4644f49501ca4c963a7fe9 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512015d2b5914fd4644f49501ca4c963a7fe9 2023-03-08 19:54:12,606 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/.tmp/family1/46aec25179034f3b8a9d7d0fbad4faea 2023-03-08 19:54:12,612 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/.tmp/family1/46aec25179034f3b8a9d7d0fbad4faea as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/46aec25179034f3b8a9d7d0fbad4faea 2023-03-08 19:54:12,616 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/46aec25179034f3b8a9d7d0fbad4faea, entries=1, sequenceid=13, filesize=4.9 K 2023-03-08 19:54:12,617 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c0d4af7836e0c3ca2f373a671b92c8f6 in 829ms, sequenceid=13, compaction requested=true 2023-03-08 19:54:12,618 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c0d4af7836e0c3ca2f373a671b92c8f6: 2023-03-08 19:54:12,618 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. 2023-03-08 19:54:12,618 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:12,618 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:12,618 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:12,618 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 19:54:12,618 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-08 19:54:12,619 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:12,619 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:12,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:12,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:12,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:12,620 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:12,620 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:12,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:12,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:12,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:12,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:12,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:12,621 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 19:54:12,621 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6cb75abf[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:12,621 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 19:54:12,621 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:12,623 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:12,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:12,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:12,623 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 19:54:12,623 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:12,623 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:12,623 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-08 19:54:12,623 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:12,625 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:12,625 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 19:54:12,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:12,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:12,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:12,625 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:12,625 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-08 19:54:12,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:12,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:12,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:12,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:12,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:12,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:12,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:12,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:12,627 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 19:54:12,627 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-08 19:54:12,627 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:12,627 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:12,627 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:12,627 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:12,628 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:12,628 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:12,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:12,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:12,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:12,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:12,628 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:12,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:12,629 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:12,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:12,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:12,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:12,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:12,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:12,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:12,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:12,630 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:12,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:12,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:12,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:12,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:12,636 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:12,636 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:12,636 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:12,636 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:12,636 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 19:54:12,636 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:12,636 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:12,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:12,636 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:12,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:12,636 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 19:54:12,636 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:12,636 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:12,637 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:12,637 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:12,637 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:12,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:12,637 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:12,637 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:12,737 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:12,738 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:12,742 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:12,744 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:12,744 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:12,744 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 19:54:12,744 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:12,744 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 19:54:12,744 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 19:54:12,745 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:12,745 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:12,746 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:12,746 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:12,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:12,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:12,747 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:12,747 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:12,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:12,747 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:12,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:12,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:12,747 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:12,748 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 19:54:12,748 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:12,748 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 19:54:12,748 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:12,748 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 19:54:12,748 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:12,748 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:12,748 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. 2023-03-08 19:54:12,748 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. started... 2023-03-08 19:54:12,748 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c0d4af7836e0c3ca2f373a671b92c8f6 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:54:13,161 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512050ca15b7e9ca846f1b74df127f46e7ae4 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512050ca15b7e9ca846f1b74df127f46e7ae4 2023-03-08 19:54:13,567 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/.tmp/family1/a5e9a47ee75d46a5b9966ef22dfa0e0a 2023-03-08 19:54:13,573 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/.tmp/family1/a5e9a47ee75d46a5b9966ef22dfa0e0a as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/a5e9a47ee75d46a5b9966ef22dfa0e0a 2023-03-08 19:54:13,577 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/a5e9a47ee75d46a5b9966ef22dfa0e0a, entries=1, sequenceid=17, filesize=4.9 K 2023-03-08 19:54:13,578 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c0d4af7836e0c3ca2f373a671b92c8f6 in 830ms, sequenceid=17, compaction requested=true 2023-03-08 19:54:13,578 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c0d4af7836e0c3ca2f373a671b92c8f6: 2023-03-08 19:54:13,578 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. 2023-03-08 19:54:13,579 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:13,579 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:13,579 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:13,579 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 19:54:13,579 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-08 19:54:13,580 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:13,580 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:13,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:13,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:13,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:13,581 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:13,581 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:13,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:13,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:13,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:13,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:13,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:13,582 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 19:54:13,582 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5df73647[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:13,582 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 19:54:13,582 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:13,583 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:13,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:13,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:13,584 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 19:54:13,584 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-08 19:54:13,584 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:13,584 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:13,584 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:13,586 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 19:54:13,586 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:13,586 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:13,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:13,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:13,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:13,586 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-08 19:54:13,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:13,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:13,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:13,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:13,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:13,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:13,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:13,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:13,588 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 19:54:13,588 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-08 19:54:13,588 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:13,588 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:13,588 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:13,588 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:13,590 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:13,590 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:13,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:13,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:13,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:13,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:13,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:13,590 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:13,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:13,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:13,590 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:13,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:13,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:13,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:13,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:13,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:13,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:13,591 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:13,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:13,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:13,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:13,593 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:13,594 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:13,594 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:13,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:13,594 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 19:54:13,594 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:13,594 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 19:54:13,594 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:13,594 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:13,594 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:13,594 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:13,594 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:13,594 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:13,594 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:13,594 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:13,594 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:13,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:13,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:13,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:13,694 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:13,695 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:13,699 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:13,701 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:13,701 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:13,701 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 19:54:13,701 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:13,701 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 19:54:13,701 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 19:54:13,701 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:13,701 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:13,702 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:13,702 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:13,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:13,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:13,703 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:13,703 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:13,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:13,703 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:13,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:13,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:13,703 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:13,704 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 19:54:13,704 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:13,704 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 19:54:13,704 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:13,704 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 19:54:13,704 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:13,704 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:13,704 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. 2023-03-08 19:54:13,704 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. started... 2023-03-08 19:54:13,704 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c0d4af7836e0c3ca2f373a671b92c8f6 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:54:14,117 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015122807ae87601b7f4e5bb14823828eb8c648 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015122807ae87601b7f4e5bb14823828eb8c648 2023-03-08 19:54:14,523 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/.tmp/family1/e5e6acd42d2b40228a0d39c0ee8e6c12 2023-03-08 19:54:14,528 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/.tmp/family1/e5e6acd42d2b40228a0d39c0ee8e6c12 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/e5e6acd42d2b40228a0d39c0ee8e6c12 2023-03-08 19:54:14,533 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/e5e6acd42d2b40228a0d39c0ee8e6c12, entries=1, sequenceid=21, filesize=4.9 K 2023-03-08 19:54:14,533 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c0d4af7836e0c3ca2f373a671b92c8f6 in 829ms, sequenceid=21, compaction requested=true 2023-03-08 19:54:14,534 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c0d4af7836e0c3ca2f373a671b92c8f6: 2023-03-08 19:54:14,534 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. 2023-03-08 19:54:14,534 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:14,534 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:14,534 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:14,534 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 19:54:14,534 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-08 19:54:14,536 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:14,536 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:14,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:14,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:14,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:14,537 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:14,537 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:14,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:14,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:14,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:14,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:14,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:14,538 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 19:54:14,538 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2d50c00f[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:14,538 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 19:54:14,538 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:14,539 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:14,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:14,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:14,539 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 19:54:14,539 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-08 19:54:14,539 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:14,539 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:14,539 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:14,541 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 19:54:14,541 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:14,541 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:14,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:14,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:14,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:14,541 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-08 19:54:14,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:14,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:14,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:14,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:14,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:14,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:14,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:14,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:14,543 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 19:54:14,543 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-08 19:54:14,543 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:14,543 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:14,543 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:14,543 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:14,544 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:14,544 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:14,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:14,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:14,545 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:14,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:14,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:14,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:14,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:14,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:14,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:14,545 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:14,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:14,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:14,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:14,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:14,546 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:14,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:14,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:14,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:14,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:14,549 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:14,549 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:14,549 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:14,549 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:14,549 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:14,549 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:14,549 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:14,549 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:14,549 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:14,549 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 19:54:14,549 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:14,549 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:14,549 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:14,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:14,549 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 19:54:14,550 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:14,550 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:14,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:14,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:14,650 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:14,651 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:14,655 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:14,657 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:14,657 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:14,657 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 19:54:14,657 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:14,657 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 19:54:14,657 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 19:54:14,657 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:14,657 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:14,658 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:14,659 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:14,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:14,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:14,659 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:14,659 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:14,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:14,659 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:14,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:14,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:14,660 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:14,660 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 19:54:14,660 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:14,660 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 19:54:14,660 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:14,660 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 19:54:14,660 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:14,660 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:14,660 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. 2023-03-08 19:54:14,660 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. started... 2023-03-08 19:54:14,660 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c0d4af7836e0c3ca2f373a671b92c8f6 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:54:14,989 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 19:54:15,074 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231a5338091ec7546e6bf716e02722d8f66 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231a5338091ec7546e6bf716e02722d8f66 2023-03-08 19:54:15,480 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/.tmp/family1/3efe5a567a8a42479a46f450177a47ec 2023-03-08 19:54:15,486 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/.tmp/family1/3efe5a567a8a42479a46f450177a47ec as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/3efe5a567a8a42479a46f450177a47ec 2023-03-08 19:54:15,491 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/3efe5a567a8a42479a46f450177a47ec, entries=1, sequenceid=25, filesize=4.9 K 2023-03-08 19:54:15,492 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c0d4af7836e0c3ca2f373a671b92c8f6 in 832ms, sequenceid=25, compaction requested=true 2023-03-08 19:54:15,492 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c0d4af7836e0c3ca2f373a671b92c8f6: 2023-03-08 19:54:15,492 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. 2023-03-08 19:54:15,492 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:15,492 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:15,492 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:15,492 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 19:54:15,492 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-08 19:54:15,494 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:15,494 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:15,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:15,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:15,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:15,495 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:15,495 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:15,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:15,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:15,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:15,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:15,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:15,496 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 19:54:15,496 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@c04b5e5[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:15,496 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 19:54:15,496 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:15,498 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:15,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:15,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:15,498 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 19:54:15,498 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:15,498 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:15,498 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-08 19:54:15,498 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:15,503 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 19:54:15,503 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:15,503 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:15,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:15,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:15,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:15,504 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-08 19:54:15,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:15,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:15,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:15,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:15,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:15,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:15,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:15,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:15,506 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 19:54:15,506 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-08 19:54:15,507 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:15,507 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:15,507 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:15,507 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:15,508 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:15,508 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:15,509 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:15,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:15,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:15,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:15,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:15,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:15,509 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:15,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:15,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:15,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:15,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:15,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:15,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:15,510 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:15,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:15,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:15,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:15,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:15,513 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:15,513 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:15,513 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:15,513 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:15,514 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 19:54:15,514 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:15,513 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:15,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:15,514 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:15,514 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:15,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:15,514 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:15,514 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:15,514 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 19:54:15,514 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:15,514 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:15,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:15,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:15,514 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:15,614 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:15,616 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:15,622 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:15,624 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:15,624 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:15,624 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 19:54:15,624 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:15,624 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 19:54:15,624 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 19:54:15,624 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:15,624 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:15,626 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:15,626 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:15,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:15,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:15,626 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:15,626 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:15,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:15,627 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:15,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:15,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:15,627 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:15,628 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 19:54:15,628 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:15,628 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 19:54:15,628 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:15,628 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 19:54:15,628 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:15,628 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:15,628 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. 2023-03-08 19:54:15,628 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. started... 2023-03-08 19:54:15,628 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c0d4af7836e0c3ca2f373a671b92c8f6 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:54:16,045 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101dd564784a88145ffbb135328769ba3e8 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101dd564784a88145ffbb135328769ba3e8 2023-03-08 19:54:16,050 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/.tmp/family1/91b558ad303c4586a9a97e193822361b 2023-03-08 19:54:16,055 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/.tmp/family1/91b558ad303c4586a9a97e193822361b as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/91b558ad303c4586a9a97e193822361b 2023-03-08 19:54:16,059 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/91b558ad303c4586a9a97e193822361b, entries=1, sequenceid=29, filesize=4.9 K 2023-03-08 19:54:16,060 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c0d4af7836e0c3ca2f373a671b92c8f6 in 432ms, sequenceid=29, compaction requested=true 2023-03-08 19:54:16,060 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c0d4af7836e0c3ca2f373a671b92c8f6: 2023-03-08 19:54:16,060 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. 2023-03-08 19:54:16,060 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:16,060 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:16,060 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,060 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 19:54:16,060 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-08 19:54:16,067 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,067 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:16,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:16,067 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,067 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:16,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:16,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:16,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:16,068 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 19:54:16,068 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5a307478[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:16,068 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 19:54:16,068 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,070 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,070 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,070 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 19:54:16,070 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-08 19:54:16,070 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:16,070 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,070 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:16,071 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,071 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 19:54:16,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:16,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:16,071 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:16,072 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-08 19:54:16,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:16,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:16,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:16,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:16,074 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 19:54:16,074 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-08 19:54:16,074 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:16,074 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:16,074 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,074 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:16,076 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,076 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:16,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,076 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:16,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:16,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:16,076 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:16,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:16,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:16,078 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:16,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,081 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,081 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:16,081 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:16,081 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:16,081 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:16,081 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 19:54:16,081 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 19:54:16,081 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:16,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:16,081 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,082 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,082 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,082 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,082 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,082 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:16,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:16,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:16,082 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:16,182 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:16,183 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:16,188 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:16,190 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:16,191 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,191 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 19:54:16,191 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:16,191 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 19:54:16,191 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 19:54:16,191 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,191 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,193 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:16,193 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:16,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:16,193 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,193 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:16,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,194 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:16,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,194 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,194 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 19:54:16,195 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,195 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 19:54:16,195 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:16,195 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 19:54:16,195 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:16,195 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:16,195 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. 2023-03-08 19:54:16,195 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. started... 2023-03-08 19:54:16,195 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c0d4af7836e0c3ca2f373a671b92c8f6 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:54:16,610 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103755bebd7180f4c4f8ea9bcd8eb97a93d to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103755bebd7180f4c4f8ea9bcd8eb97a93d 2023-03-08 19:54:16,616 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/.tmp/family1/e688baaf23f3418b9d17182f4d0944e4 2023-03-08 19:54:16,622 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/.tmp/family1/e688baaf23f3418b9d17182f4d0944e4 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/e688baaf23f3418b9d17182f4d0944e4 2023-03-08 19:54:16,627 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/e688baaf23f3418b9d17182f4d0944e4, entries=1, sequenceid=33, filesize=4.9 K 2023-03-08 19:54:16,628 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c0d4af7836e0c3ca2f373a671b92c8f6 in 433ms, sequenceid=33, compaction requested=true 2023-03-08 19:54:16,628 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c0d4af7836e0c3ca2f373a671b92c8f6: 2023-03-08 19:54:16,629 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. 2023-03-08 19:54:16,629 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:16,629 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:16,629 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,629 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 19:54:16,629 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-08 19:54:16,631 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,631 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:16,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:16,631 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:16,631 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:16,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:16,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:16,633 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 19:54:16,633 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3e913477[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:16,633 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 19:54:16,633 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,634 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,635 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 19:54:16,635 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,635 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:16,635 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-08 19:54:16,635 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:16,637 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,637 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 19:54:16,637 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:16,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:16,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:16,637 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-08 19:54:16,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:16,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:16,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:16,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:16,640 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 19:54:16,640 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-08 19:54:16,640 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:16,640 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:16,640 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,640 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:16,642 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,642 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,642 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:16,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:16,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:16,642 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:16,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:16,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:16,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:16,644 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:16,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,646 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,646 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,646 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,646 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:16,646 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:16,647 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 19:54:16,647 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:16,647 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 19:54:16,647 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:16,647 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:16,647 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,647 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:16,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:16,647 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:16,647 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,647 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:16,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:16,747 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:16,749 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:16,753 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:16,755 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:16,755 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,755 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 19:54:16,755 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:16,756 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 19:54:16,756 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 19:54:16,756 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,756 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,758 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:16,758 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:16,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:16,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,758 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,758 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:16,758 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:16,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,759 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,759 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 19:54:16,759 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,759 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 19:54:16,759 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:16,759 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 19:54:16,760 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:16,760 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:16,760 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. 2023-03-08 19:54:16,760 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. started... 2023-03-08 19:54:16,760 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c0d4af7836e0c3ca2f373a671b92c8f6 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:54:16,779 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907df49ca6d26de46578b67d43eabea0092 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20150907df49ca6d26de46578b67d43eabea0092 2023-03-08 19:54:16,790 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/.tmp/family1/6a66877b9a1f41edaa1f30e4e5787f12 2023-03-08 19:54:16,795 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/.tmp/family1/6a66877b9a1f41edaa1f30e4e5787f12 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/6a66877b9a1f41edaa1f30e4e5787f12 2023-03-08 19:54:16,803 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/6a66877b9a1f41edaa1f30e4e5787f12, entries=1, sequenceid=37, filesize=4.9 K 2023-03-08 19:54:16,804 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c0d4af7836e0c3ca2f373a671b92c8f6 in 44ms, sequenceid=37, compaction requested=true 2023-03-08 19:54:16,804 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c0d4af7836e0c3ca2f373a671b92c8f6: 2023-03-08 19:54:16,804 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. 2023-03-08 19:54:16,804 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:16,805 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:16,805 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,805 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 19:54:16,805 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-08 19:54:16,808 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,808 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:16,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:16,808 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,808 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:16,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:16,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:16,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:16,810 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 19:54:16,810 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@15eacb12[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:16,810 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 19:54:16,810 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,815 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,815 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 19:54:16,815 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-08 19:54:16,815 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,815 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:16,815 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:16,817 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 19:54:16,817 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:16,817 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,817 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-08 19:54:16,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:16,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:16,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:16,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:16,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:16,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:16,820 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 19:54:16,820 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-08 19:54:16,820 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:16,820 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:16,820 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,820 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:16,823 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,824 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:16,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,824 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:16,824 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:16,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:16,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:16,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:16,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:16,826 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:16,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,837 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,837 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,837 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:16,837 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:16,837 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:16,837 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 19:54:16,837 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 19:54:16,837 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,837 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:16,837 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:16,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:16,837 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,838 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,838 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,838 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,838 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:16,838 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:16,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:16,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:16,938 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:16,939 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:16,944 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:16,946 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:16,946 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,946 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 19:54:16,946 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:16,946 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 19:54:16,947 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 19:54:16,947 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,947 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,950 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:16,950 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:16,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:16,950 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:16,950 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:16,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,950 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:16,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,951 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,951 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 19:54:16,951 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:16,951 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 19:54:16,951 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:16,951 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 19:54:16,952 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:16,952 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:16,952 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. 2023-03-08 19:54:16,952 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. started... 2023-03-08 19:54:16,952 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c0d4af7836e0c3ca2f373a671b92c8f6 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:54:17,376 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112073df94f11809457f83852cf6a8dd5f76 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015112073df94f11809457f83852cf6a8dd5f76 2023-03-08 19:54:17,383 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/.tmp/family1/045672821df64164a875959721c023b0 2023-03-08 19:54:17,389 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/.tmp/family1/045672821df64164a875959721c023b0 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/045672821df64164a875959721c023b0 2023-03-08 19:54:17,404 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/045672821df64164a875959721c023b0, entries=1, sequenceid=41, filesize=4.9 K 2023-03-08 19:54:17,406 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c0d4af7836e0c3ca2f373a671b92c8f6 in 454ms, sequenceid=41, compaction requested=true 2023-03-08 19:54:17,406 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c0d4af7836e0c3ca2f373a671b92c8f6: 2023-03-08 19:54:17,406 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. 2023-03-08 19:54:17,406 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:17,407 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:17,407 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:17,407 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 19:54:17,407 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-08 19:54:17,408 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:17,409 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:17,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:17,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:17,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:17,409 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:17,409 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:17,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:17,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:17,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:17,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:17,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:17,411 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 19:54:17,411 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3c68efd7[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:17,411 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 19:54:17,411 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:17,413 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:17,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:17,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:17,413 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 19:54:17,413 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-08 19:54:17,413 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:17,413 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:17,413 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:17,415 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:17,415 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 19:54:17,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:17,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:17,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:17,415 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:17,415 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-08 19:54:17,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:17,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:17,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:17,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:17,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:17,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:17,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:17,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:17,417 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 19:54:17,417 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-08 19:54:17,418 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:17,418 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:17,418 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:17,418 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:17,419 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:17,419 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:17,419 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:17,419 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:17,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:17,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:17,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:17,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:17,420 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:17,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:17,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:17,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:17,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:17,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:17,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:17,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:17,421 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:17,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:17,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:17,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:17,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:17,423 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:17,423 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:17,423 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:17,424 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:17,424 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-08 19:54:17,424 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:17,423 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:17,424 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:17,424 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:17,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:17,424 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:17,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:17,424 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 19:54:17,424 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:17,424 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:17,425 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:17,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:17,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:17,425 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:17,525 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:17,526 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:17,528 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39989] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMinorCompactionWithWeeklyPolicy 2023-03-08 19:54:17,529 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] procedure2.ProcedureExecutor(1029): Stored pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 19:54:17,530 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 19:54:17,531 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 19:54:17,531 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=26, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678305857531 2023-03-08 19:54:17,533 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-08 19:54:17,534 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 0 del files, 9 selected files, and 0 irrelevant files table 'testMinorCompactionWithWeeklyPolicy' and column 'family1' 2023-03-08 19:54:17,534 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 19:54:17,534 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] zookeeper.ReadOnlyZKClient(139): Connect 0x64c3cb99 to 127.0.0.1:56751 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 19:54:17,545 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@16c4efef, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 19:54:17,545 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 19:54:17,607 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601031f83df1bac4c433396ec0f946d15dfbe to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601031f83df1bac4c433396ec0f946d15dfbe 2023-03-08 19:54:17,608 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 19:54:17,627 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 19:54:17,629 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60284, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 19:54:17,638 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 19:54:17,641 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58656, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 19:54:17,677 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/694ad5fefb094885bd9c6ad98b568a1b first=Optional[r3] last=Optional[r6] 2023-03-08 19:54:17,678 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6., hostname=jenkins-hbase4.apache.org,33367,1678305218680, seqNum=2 for row with hfile group [{family1,hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/694ad5fefb094885bd9c6ad98b568a1b}] 2023-03-08 19:54:17,687 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(763): Validating hfile at hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/694ad5fefb094885bd9c6ad98b568a1b for inclusion in c0d4af7836e0c3ca2f373a671b92c8f6/family1 2023-03-08 19:54:17,691 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-08 19:54:17,691 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 19:54:17,691 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(2446): Flush status journal for c0d4af7836e0c3ca2f373a671b92c8f6: 2023-03-08 19:54:17,692 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/694ad5fefb094885bd9c6ad98b568a1b to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMinorCompactionWithWeeklyPolicy__ja4b0947t2ktt4ucupdq4i0u6792p0otn7cvog4980ijoqupfjrutem2ar4nuqld/family1/694ad5fefb094885bd9c6ad98b568a1b 2023-03-08 19:54:17,695 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMinorCompactionWithWeeklyPolicy__ja4b0947t2ktt4ucupdq4i0u6792p0otn7cvog4980ijoqupfjrutem2ar4nuqld/family1/694ad5fefb094885bd9c6ad98b568a1b as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/5406ca233c9e4e5ab1dbff335e404d93_SeqId_44_ 2023-03-08 19:54:17,696 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(849): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMinorCompactionWithWeeklyPolicy__ja4b0947t2ktt4ucupdq4i0u6792p0otn7cvog4980ijoqupfjrutem2ar4nuqld/family1/694ad5fefb094885bd9c6ad98b568a1b into c0d4af7836e0c3ca2f373a671b92c8f6/family1 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/5406ca233c9e4e5ab1dbff335e404d93_SeqId_44_ - updating store file list. 2023-03-08 19:54:17,701 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(882): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/5406ca233c9e4e5ab1dbff335e404d93_SeqId_44_ into c0d4af7836e0c3ca2f373a671b92c8f6/family1 2023-03-08 19:54:17,701 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(855): Successfully loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMinorCompactionWithWeeklyPolicy__ja4b0947t2ktt4ucupdq4i0u6792p0otn7cvog4980ijoqupfjrutem2ar4nuqld/family1/694ad5fefb094885bd9c6ad98b568a1b into c0d4af7836e0c3ca2f373a671b92c8f6/family1 (new location: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/5406ca233c9e4e5ab1dbff335e404d93_SeqId_44_) 2023-03-08 19:54:17,702 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMinorCompactionWithWeeklyPolicy__ja4b0947t2ktt4ucupdq4i0u6792p0otn7cvog4980ijoqupfjrutem2ar4nuqld/family1/694ad5fefb094885bd9c6ad98b568a1b 2023-03-08 19:54:17,702 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-08 19:54:17,705 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-08 19:54:17,705 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(1912): c0d4af7836e0c3ca2f373a671b92c8f6/family1 is initiating minor compaction 2023-03-08 19:54:17,706 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6edd8cc6; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 19:54:17,706 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of c0d4af7836e0c3ca2f373a671b92c8f6/family1 in testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. 2023-03-08 19:54:17,706 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/88e43825322d446a9f8465cc968de562, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/46aec25179034f3b8a9d7d0fbad4faea, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/a5e9a47ee75d46a5b9966ef22dfa0e0a, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/e5e6acd42d2b40228a0d39c0ee8e6c12, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/3efe5a567a8a42479a46f450177a47ec, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/91b558ad303c4586a9a97e193822361b, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/e688baaf23f3418b9d17182f4d0944e4, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/6a66877b9a1f41edaa1f30e4e5787f12, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/045672821df64164a875959721c023b0, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/5406ca233c9e4e5ab1dbff335e404d93_SeqId_44_] into tmpdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/.tmp, totalSize=50.0 K 2023-03-08 19:54:17,706 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 88e43825322d446a9f8465cc968de562, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-08 19:54:17,706 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 46aec25179034f3b8a9d7d0fbad4faea, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-08 19:54:17,706 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting a5e9a47ee75d46a5b9966ef22dfa0e0a, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-08 19:54:17,706 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting e5e6acd42d2b40228a0d39c0ee8e6c12, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-08 19:54:17,706 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 3efe5a567a8a42479a46f450177a47ec, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-08 19:54:17,706 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 91b558ad303c4586a9a97e193822361b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-08 19:54:17,706 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting e688baaf23f3418b9d17182f4d0944e4, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-08 19:54:17,707 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 6a66877b9a1f41edaa1f30e4e5787f12, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-08 19:54:17,707 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 045672821df64164a875959721c023b0, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-08 19:54:17,707 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 5406ca233c9e4e5ab1dbff335e404d93_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-08 19:54:17,708 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 19:54:17,709 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:54:17,711 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015122807ae87601b7f4e5bb14823828eb8c648 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015122807ae87601b7f4e5bb14823828eb8c648 2023-03-08 19:54:17,712 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231a5338091ec7546e6bf716e02722d8f66 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231a5338091ec7546e6bf716e02722d8f66 2023-03-08 19:54:17,714 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101dd564784a88145ffbb135328769ba3e8 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101dd564784a88145ffbb135328769ba3e8 2023-03-08 19:54:17,717 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103755bebd7180f4c4f8ea9bcd8eb97a93d to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103755bebd7180f4c4f8ea9bcd8eb97a93d 2023-03-08 19:54:17,717 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-03-08 19:54:17,717 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-08 19:54:17,717 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 19:54:17,717 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 19:54:17,718 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 19:54:17,718 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 19:54:17,718 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 19:54:17,718 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 19:54:17,718 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 19:54:17,718 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 19:54:17,776 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] throttle.PressureAwareThroughputController(145): c0d4af7836e0c3ca2f373a671b92c8f6#family1#compaction#34 average throughput is 0.34 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 19:54:18,204 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/.tmp/family1/9d9884b58c7342f38eea9acf80168677 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/9d9884b58c7342f38eea9acf80168677 2023-03-08 19:54:18,211 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in c0d4af7836e0c3ca2f373a671b92c8f6/family1 of c0d4af7836e0c3ca2f373a671b92c8f6 into 9d9884b58c7342f38eea9acf80168677(size=6.6 K), total size for store is 171.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 19:54:18,212 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for c0d4af7836e0c3ca2f373a671b92c8f6: 2023-03-08 19:54:18,212 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1678305257705; duration=0sec 2023-03-08 19:54:18,212 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 19:54:18,232 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/88e43825322d446a9f8465cc968de562, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/46aec25179034f3b8a9d7d0fbad4faea, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/a5e9a47ee75d46a5b9966ef22dfa0e0a, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/e5e6acd42d2b40228a0d39c0ee8e6c12, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/3efe5a567a8a42479a46f450177a47ec, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/91b558ad303c4586a9a97e193822361b, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/e688baaf23f3418b9d17182f4d0944e4, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/6a66877b9a1f41edaa1f30e4e5787f12, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/045672821df64164a875959721c023b0, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/5406ca233c9e4e5ab1dbff335e404d93_SeqId_44_] to archive 2023-03-08 19:54:18,234 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:54:18,236 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/88e43825322d446a9f8465cc968de562 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/88e43825322d446a9f8465cc968de562 2023-03-08 19:54:18,237 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/46aec25179034f3b8a9d7d0fbad4faea to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/46aec25179034f3b8a9d7d0fbad4faea 2023-03-08 19:54:18,239 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/a5e9a47ee75d46a5b9966ef22dfa0e0a to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/a5e9a47ee75d46a5b9966ef22dfa0e0a 2023-03-08 19:54:18,240 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/e5e6acd42d2b40228a0d39c0ee8e6c12 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/e5e6acd42d2b40228a0d39c0ee8e6c12 2023-03-08 19:54:18,241 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/3efe5a567a8a42479a46f450177a47ec to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/3efe5a567a8a42479a46f450177a47ec 2023-03-08 19:54:18,244 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/91b558ad303c4586a9a97e193822361b to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/91b558ad303c4586a9a97e193822361b 2023-03-08 19:54:18,247 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/e688baaf23f3418b9d17182f4d0944e4 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/e688baaf23f3418b9d17182f4d0944e4 2023-03-08 19:54:18,249 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/6a66877b9a1f41edaa1f30e4e5787f12 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/6a66877b9a1f41edaa1f30e4e5787f12 2023-03-08 19:54:18,250 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/045672821df64164a875959721c023b0 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/045672821df64164a875959721c023b0 2023-03-08 19:54:18,252 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/5406ca233c9e4e5ab1dbff335e404d93_SeqId_44_ to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/5406ca233c9e4e5ab1dbff335e404d93_SeqId_44_ 2023-03-08 19:54:18,575 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120530bf033b493d48479f04cfa0e1f7752d to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120530bf033b493d48479f04cfa0e1f7752d 2023-03-08 19:54:18,576 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 19:54:18,601 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/a9622f01b15c4d84bcb628575629f498 first=Optional[r1] last=Optional[r2] 2023-03-08 19:54:18,602 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6., hostname=jenkins-hbase4.apache.org,33367,1678305218680, seqNum=2 for row with hfile group [{family1,hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/a9622f01b15c4d84bcb628575629f498}] 2023-03-08 19:54:18,606 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(763): Validating hfile at hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/a9622f01b15c4d84bcb628575629f498 for inclusion in c0d4af7836e0c3ca2f373a671b92c8f6/family1 2023-03-08 19:54:18,612 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-03-08 19:54:18,612 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 19:54:18,612 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(2446): Flush status journal for c0d4af7836e0c3ca2f373a671b92c8f6: 2023-03-08 19:54:18,613 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/a9622f01b15c4d84bcb628575629f498 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMinorCompactionWithWeeklyPolicy__pjduvr7vk340egnj384tr27g0rkkcf5mjf2uejqcl92q2v02s540vj7pcanuou3h/family1/a9622f01b15c4d84bcb628575629f498 2023-03-08 19:54:18,616 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMinorCompactionWithWeeklyPolicy__pjduvr7vk340egnj384tr27g0rkkcf5mjf2uejqcl92q2v02s540vj7pcanuou3h/family1/a9622f01b15c4d84bcb628575629f498 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/170e924c307246fabbb5989f5f488a68_SeqId_47_ 2023-03-08 19:54:18,616 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(849): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMinorCompactionWithWeeklyPolicy__pjduvr7vk340egnj384tr27g0rkkcf5mjf2uejqcl92q2v02s540vj7pcanuou3h/family1/a9622f01b15c4d84bcb628575629f498 into c0d4af7836e0c3ca2f373a671b92c8f6/family1 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/170e924c307246fabbb5989f5f488a68_SeqId_47_ - updating store file list. 2023-03-08 19:54:18,621 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(882): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/170e924c307246fabbb5989f5f488a68_SeqId_47_ into c0d4af7836e0c3ca2f373a671b92c8f6/family1 2023-03-08 19:54:18,622 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(855): Successfully loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMinorCompactionWithWeeklyPolicy__pjduvr7vk340egnj384tr27g0rkkcf5mjf2uejqcl92q2v02s540vj7pcanuou3h/family1/a9622f01b15c4d84bcb628575629f498 into c0d4af7836e0c3ca2f373a671b92c8f6/family1 (new location: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/170e924c307246fabbb5989f5f488a68_SeqId_47_) 2023-03-08 19:54:18,622 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMinorCompactionWithWeeklyPolicy__pjduvr7vk340egnj384tr27g0rkkcf5mjf2uejqcl92q2v02s540vj7pcanuou3h/family1/a9622f01b15c4d84bcb628575629f498 2023-03-08 19:54:18,622 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-08 19:54:18,624 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 181014 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-08 19:54:18,624 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(1912): c0d4af7836e0c3ca2f373a671b92c8f6/family1 is initiating minor compaction (all files) 2023-03-08 19:54:18,624 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@62e7aecb; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 19:54:18,624 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of c0d4af7836e0c3ca2f373a671b92c8f6/family1 in testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. 2023-03-08 19:54:18,624 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/943c65840cfa44a8932da0ff713746be, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/9d9884b58c7342f38eea9acf80168677, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/170e924c307246fabbb5989f5f488a68_SeqId_47_] into tmpdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/.tmp, totalSize=176.8 K 2023-03-08 19:54:18,625 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 943c65840cfa44a8932da0ff713746be, keycount=1000, bloomtype=ROW, size=165.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878843488 2023-03-08 19:54:18,626 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 9d9884b58c7342f38eea9acf80168677, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621243488 2023-03-08 19:54:18,626 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 19:54:18,626 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 170e924c307246fabbb5989f5f488a68_SeqId_47_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965243488 2023-03-08 19:54:18,627 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:54:18,630 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512015d2b5914fd4644f49501ca4c963a7fe9 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512015d2b5914fd4644f49501ca4c963a7fe9 2023-03-08 19:54:18,631 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512050ca15b7e9ca846f1b74df127f46e7ae4 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512050ca15b7e9ca846f1b74df127f46e7ae4 2023-03-08 19:54:18,631 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 2 expired mob files are deleted 2023-03-08 19:54:18,631 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-08 19:54:18,632 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 19:54:18,632 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 19:54:18,633 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 19:54:18,634 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=26, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 19:54:18,636 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.1060 sec 2023-03-08 19:54:18,654 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] throttle.PressureAwareThroughputController(145): c0d4af7836e0c3ca2f373a671b92c8f6#family1#compaction#35 average throughput is 15.46 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 19:54:18,759 INFO [Listener at localhost/34049] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=257 (was 250) - Thread LEAK? -, OpenFileDescriptor=603 (was 595) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=219 (was 250), ProcessCount=170 (was 170), AvailableMemoryMB=8485 (was 8826) 2023-03-08 19:54:18,775 INFO [Listener at localhost/34049] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=257, OpenFileDescriptor=603, MaxFileDescriptor=60000, SystemLoadAverage=219, ProcessCount=170, AvailableMemoryMB=8484 2023-03-08 19:54:18,778 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-08 19:54:18,779 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure2.ProcedureExecutor(1029): Stored pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:18,781 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 19:54:18,781 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testScannerOnBulkLoadRefHFiles" procId is: 27 2023-03-08 19:54:18,782 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-08 19:54:18,782 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 19:54:18,785 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testScannerOnBulkLoadRefHFiles/b4dc975db9245bfec2f0c0dc6879a14e 2023-03-08 19:54:18,785 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testScannerOnBulkLoadRefHFiles/2232a25e9ac1db961952c09e8e81f27b 2023-03-08 19:54:18,785 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testScannerOnBulkLoadRefHFiles/f43229ba0f524b69e8eb52fe67361236 2023-03-08 19:54:18,785 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testScannerOnBulkLoadRefHFiles/2232a25e9ac1db961952c09e8e81f27b empty. 2023-03-08 19:54:18,785 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testScannerOnBulkLoadRefHFiles/b4dc975db9245bfec2f0c0dc6879a14e empty. 2023-03-08 19:54:18,786 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testScannerOnBulkLoadRefHFiles/f43229ba0f524b69e8eb52fe67361236 empty. 2023-03-08 19:54:18,786 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testScannerOnBulkLoadRefHFiles/2232a25e9ac1db961952c09e8e81f27b 2023-03-08 19:54:18,786 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testScannerOnBulkLoadRefHFiles/b4dc975db9245bfec2f0c0dc6879a14e 2023-03-08 19:54:18,786 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testScannerOnBulkLoadRefHFiles/f43229ba0f524b69e8eb52fe67361236 2023-03-08 19:54:18,786 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testScannerOnBulkLoadRefHFiles regions 2023-03-08 19:54:18,798 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testScannerOnBulkLoadRefHFiles/.tabledesc/.tableinfo.0000000001 2023-03-08 19:54:18,800 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(7675): creating {ENCODED => f43229ba0f524b69e8eb52fe67361236, NAME => 'testScannerOnBulkLoadRefHFiles,2,1678305258777.f43229ba0f524b69e8eb52fe67361236.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp 2023-03-08 19:54:18,800 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(7675): creating {ENCODED => b4dc975db9245bfec2f0c0dc6879a14e, NAME => 'testScannerOnBulkLoadRefHFiles,,1678305258777.b4dc975db9245bfec2f0c0dc6879a14e.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp 2023-03-08 19:54:18,800 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(7675): creating {ENCODED => 2232a25e9ac1db961952c09e8e81f27b, NAME => 'testScannerOnBulkLoadRefHFiles,1,1678305258777.2232a25e9ac1db961952c09e8e81f27b.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp 2023-03-08 19:54:18,818 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,1,1678305258777.2232a25e9ac1db961952c09e8e81f27b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:54:18,818 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1604): Closing 2232a25e9ac1db961952c09e8e81f27b, disabling compactions & flushes 2023-03-08 19:54:18,818 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,1,1678305258777.2232a25e9ac1db961952c09e8e81f27b. 2023-03-08 19:54:18,818 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1678305258777.2232a25e9ac1db961952c09e8e81f27b. 2023-03-08 19:54:18,818 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1678305258777.2232a25e9ac1db961952c09e8e81f27b. after waiting 0 ms 2023-03-08 19:54:18,818 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1678305258777.2232a25e9ac1db961952c09e8e81f27b. 2023-03-08 19:54:18,818 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,1,1678305258777.2232a25e9ac1db961952c09e8e81f27b. 2023-03-08 19:54:18,818 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1558): Region close journal for 2232a25e9ac1db961952c09e8e81f27b: 2023-03-08 19:54:18,822 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,,1678305258777.b4dc975db9245bfec2f0c0dc6879a14e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:54:18,822 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1604): Closing b4dc975db9245bfec2f0c0dc6879a14e, disabling compactions & flushes 2023-03-08 19:54:18,823 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,,1678305258777.b4dc975db9245bfec2f0c0dc6879a14e. 2023-03-08 19:54:18,823 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1678305258777.b4dc975db9245bfec2f0c0dc6879a14e. 2023-03-08 19:54:18,823 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1678305258777.b4dc975db9245bfec2f0c0dc6879a14e. after waiting 0 ms 2023-03-08 19:54:18,823 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1678305258777.b4dc975db9245bfec2f0c0dc6879a14e. 2023-03-08 19:54:18,823 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,,1678305258777.b4dc975db9245bfec2f0c0dc6879a14e. 2023-03-08 19:54:18,823 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1558): Region close journal for b4dc975db9245bfec2f0c0dc6879a14e: 2023-03-08 19:54:18,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-08 19:54:19,085 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-08 19:54:19,220 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,2,1678305258777.f43229ba0f524b69e8eb52fe67361236.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:54:19,221 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1604): Closing f43229ba0f524b69e8eb52fe67361236, disabling compactions & flushes 2023-03-08 19:54:19,221 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,2,1678305258777.f43229ba0f524b69e8eb52fe67361236. 2023-03-08 19:54:19,221 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1678305258777.f43229ba0f524b69e8eb52fe67361236. 2023-03-08 19:54:19,221 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1678305258777.f43229ba0f524b69e8eb52fe67361236. after waiting 0 ms 2023-03-08 19:54:19,221 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1678305258777.f43229ba0f524b69e8eb52fe67361236. 2023-03-08 19:54:19,221 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,2,1678305258777.f43229ba0f524b69e8eb52fe67361236. 2023-03-08 19:54:19,221 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1558): Region close journal for f43229ba0f524b69e8eb52fe67361236: 2023-03-08 19:54:19,223 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 19:54:19,224 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,1,1678305258777.2232a25e9ac1db961952c09e8e81f27b.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1678305259224"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678305259224"}]},"ts":"1678305259224"} 2023-03-08 19:54:19,225 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,,1678305258777.b4dc975db9245bfec2f0c0dc6879a14e.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678305259224"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678305259224"}]},"ts":"1678305259224"} 2023-03-08 19:54:19,225 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,2,1678305258777.f43229ba0f524b69e8eb52fe67361236.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678305259224"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678305259224"}]},"ts":"1678305259224"} 2023-03-08 19:54:19,228 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-08 19:54:19,229 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 19:54:19,229 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678305259229"}]},"ts":"1678305259229"} 2023-03-08 19:54:19,230 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLING in hbase:meta 2023-03-08 19:54:19,233 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=b4dc975db9245bfec2f0c0dc6879a14e, ASSIGN}, {pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=2232a25e9ac1db961952c09e8e81f27b, ASSIGN}, {pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=f43229ba0f524b69e8eb52fe67361236, ASSIGN}] 2023-03-08 19:54:19,235 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=b4dc975db9245bfec2f0c0dc6879a14e, ASSIGN 2023-03-08 19:54:19,235 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=2232a25e9ac1db961952c09e8e81f27b, ASSIGN 2023-03-08 19:54:19,236 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=f43229ba0f524b69e8eb52fe67361236, ASSIGN 2023-03-08 19:54:19,236 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=2232a25e9ac1db961952c09e8e81f27b, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33367,1678305218680; forceNewPlan=false, retain=false 2023-03-08 19:54:19,236 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=b4dc975db9245bfec2f0c0dc6879a14e, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33367,1678305218680; forceNewPlan=false, retain=false 2023-03-08 19:54:19,237 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=f43229ba0f524b69e8eb52fe67361236, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33367,1678305218680; forceNewPlan=false, retain=false 2023-03-08 19:54:19,386 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-08 19:54:19,389 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=b4dc975db9245bfec2f0c0dc6879a14e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:19,389 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=f43229ba0f524b69e8eb52fe67361236, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:19,389 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=2232a25e9ac1db961952c09e8e81f27b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:19,389 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,,1678305258777.b4dc975db9245bfec2f0c0dc6879a14e.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678305259388"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678305259388"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678305259388"}]},"ts":"1678305259388"} 2023-03-08 19:54:19,389 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,2,1678305258777.f43229ba0f524b69e8eb52fe67361236.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678305259388"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678305259388"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678305259388"}]},"ts":"1678305259388"} 2023-03-08 19:54:19,389 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,1,1678305258777.2232a25e9ac1db961952c09e8e81f27b.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1678305259388"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678305259388"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678305259388"}]},"ts":"1678305259388"} 2023-03-08 19:54:19,391 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE; OpenRegionProcedure b4dc975db9245bfec2f0c0dc6879a14e, server=jenkins-hbase4.apache.org,33367,1678305218680}] 2023-03-08 19:54:19,392 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=32, ppid=30, state=RUNNABLE; OpenRegionProcedure f43229ba0f524b69e8eb52fe67361236, server=jenkins-hbase4.apache.org,33367,1678305218680}] 2023-03-08 19:54:19,392 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=29, state=RUNNABLE; OpenRegionProcedure 2232a25e9ac1db961952c09e8e81f27b, server=jenkins-hbase4.apache.org,33367,1678305218680}] 2023-03-08 19:54:19,486 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/.tmp/family1/11dcf25e7ccd4939a2c4fffa19b7b55c as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/11dcf25e7ccd4939a2c4fffa19b7b55c 2023-03-08 19:54:19,491 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in c0d4af7836e0c3ca2f373a671b92c8f6/family1 of c0d4af7836e0c3ca2f373a671b92c8f6 into 11dcf25e7ccd4939a2c4fffa19b7b55c(size=166.4 K), total size for store is 166.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 19:54:19,491 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for c0d4af7836e0c3ca2f373a671b92c8f6: 2023-03-08 19:54:19,491 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6., storeName=family1, fileCount=3, fileSize=176.8 K (165.1 K, 6.6 K, 5.2 K), priority=2, time=1678305258624; duration=0sec 2023-03-08 19:54:19,491 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 19:54:19,516 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/943c65840cfa44a8932da0ff713746be, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/9d9884b58c7342f38eea9acf80168677, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/170e924c307246fabbb5989f5f488a68_SeqId_47_] to archive 2023-03-08 19:54:19,517 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:54:19,519 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/943c65840cfa44a8932da0ff713746be to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/943c65840cfa44a8932da0ff713746be 2023-03-08 19:54:19,520 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/9d9884b58c7342f38eea9acf80168677 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/9d9884b58c7342f38eea9acf80168677 2023-03-08 19:54:19,521 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/170e924c307246fabbb5989f5f488a68_SeqId_47_ to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/family1/170e924c307246fabbb5989f5f488a68_SeqId_47_ 2023-03-08 19:54:19,544 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,1,1678305258777.2232a25e9ac1db961952c09e8e81f27b. 2023-03-08 19:54:19,545 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2232a25e9ac1db961952c09e8e81f27b, NAME => 'testScannerOnBulkLoadRefHFiles,1,1678305258777.2232a25e9ac1db961952c09e8e81f27b.', STARTKEY => '1', ENDKEY => '2'} 2023-03-08 19:54:19,545 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 2232a25e9ac1db961952c09e8e81f27b 2023-03-08 19:54:19,545 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,1,1678305258777.2232a25e9ac1db961952c09e8e81f27b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:54:19,545 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2232a25e9ac1db961952c09e8e81f27b 2023-03-08 19:54:19,545 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2232a25e9ac1db961952c09e8e81f27b 2023-03-08 19:54:19,546 INFO [StoreOpener-2232a25e9ac1db961952c09e8e81f27b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 2232a25e9ac1db961952c09e8e81f27b 2023-03-08 19:54:19,548 DEBUG [StoreOpener-2232a25e9ac1db961952c09e8e81f27b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/2232a25e9ac1db961952c09e8e81f27b/family1 2023-03-08 19:54:19,548 DEBUG [StoreOpener-2232a25e9ac1db961952c09e8e81f27b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/2232a25e9ac1db961952c09e8e81f27b/family1 2023-03-08 19:54:19,548 INFO [StoreOpener-2232a25e9ac1db961952c09e8e81f27b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2232a25e9ac1db961952c09e8e81f27b columnFamilyName family1 2023-03-08 19:54:19,550 INFO [StoreOpener-2232a25e9ac1db961952c09e8e81f27b-1] regionserver.HStore(310): Store=2232a25e9ac1db961952c09e8e81f27b/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:54:19,550 INFO [StoreOpener-2232a25e9ac1db961952c09e8e81f27b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 2232a25e9ac1db961952c09e8e81f27b 2023-03-08 19:54:19,551 DEBUG [StoreOpener-2232a25e9ac1db961952c09e8e81f27b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/2232a25e9ac1db961952c09e8e81f27b/family2 2023-03-08 19:54:19,551 DEBUG [StoreOpener-2232a25e9ac1db961952c09e8e81f27b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/2232a25e9ac1db961952c09e8e81f27b/family2 2023-03-08 19:54:19,552 INFO [StoreOpener-2232a25e9ac1db961952c09e8e81f27b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2232a25e9ac1db961952c09e8e81f27b columnFamilyName family2 2023-03-08 19:54:19,553 INFO [StoreOpener-2232a25e9ac1db961952c09e8e81f27b-1] regionserver.HStore(310): Store=2232a25e9ac1db961952c09e8e81f27b/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:54:19,554 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/2232a25e9ac1db961952c09e8e81f27b 2023-03-08 19:54:19,554 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/2232a25e9ac1db961952c09e8e81f27b 2023-03-08 19:54:19,556 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 19:54:19,557 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2232a25e9ac1db961952c09e8e81f27b 2023-03-08 19:54:19,558 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/2232a25e9ac1db961952c09e8e81f27b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 19:54:19,559 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2232a25e9ac1db961952c09e8e81f27b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66328496, jitterRate=-0.011628389358520508}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 19:54:19,559 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2232a25e9ac1db961952c09e8e81f27b: 2023-03-08 19:54:19,560 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,1,1678305258777.2232a25e9ac1db961952c09e8e81f27b., pid=33, masterSystemTime=1678305259542 2023-03-08 19:54:19,561 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,1,1678305258777.2232a25e9ac1db961952c09e8e81f27b. 2023-03-08 19:54:19,561 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,1,1678305258777.2232a25e9ac1db961952c09e8e81f27b. 2023-03-08 19:54:19,561 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,,1678305258777.b4dc975db9245bfec2f0c0dc6879a14e. 2023-03-08 19:54:19,561 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b4dc975db9245bfec2f0c0dc6879a14e, NAME => 'testScannerOnBulkLoadRefHFiles,,1678305258777.b4dc975db9245bfec2f0c0dc6879a14e.', STARTKEY => '', ENDKEY => '1'} 2023-03-08 19:54:19,562 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=2232a25e9ac1db961952c09e8e81f27b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:19,562 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles b4dc975db9245bfec2f0c0dc6879a14e 2023-03-08 19:54:19,562 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,,1678305258777.b4dc975db9245bfec2f0c0dc6879a14e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:54:19,562 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,1,1678305258777.2232a25e9ac1db961952c09e8e81f27b.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1678305259562"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678305259562"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678305259562"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678305259562"}]},"ts":"1678305259562"} 2023-03-08 19:54:19,562 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b4dc975db9245bfec2f0c0dc6879a14e 2023-03-08 19:54:19,562 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b4dc975db9245bfec2f0c0dc6879a14e 2023-03-08 19:54:19,563 INFO [StoreOpener-b4dc975db9245bfec2f0c0dc6879a14e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region b4dc975db9245bfec2f0c0dc6879a14e 2023-03-08 19:54:19,565 DEBUG [StoreOpener-b4dc975db9245bfec2f0c0dc6879a14e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/b4dc975db9245bfec2f0c0dc6879a14e/family1 2023-03-08 19:54:19,565 DEBUG [StoreOpener-b4dc975db9245bfec2f0c0dc6879a14e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/b4dc975db9245bfec2f0c0dc6879a14e/family1 2023-03-08 19:54:19,565 INFO [StoreOpener-b4dc975db9245bfec2f0c0dc6879a14e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b4dc975db9245bfec2f0c0dc6879a14e columnFamilyName family1 2023-03-08 19:54:19,565 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=29 2023-03-08 19:54:19,566 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=29, state=SUCCESS; OpenRegionProcedure 2232a25e9ac1db961952c09e8e81f27b, server=jenkins-hbase4.apache.org,33367,1678305218680 in 171 msec 2023-03-08 19:54:19,567 INFO [StoreOpener-b4dc975db9245bfec2f0c0dc6879a14e-1] regionserver.HStore(310): Store=b4dc975db9245bfec2f0c0dc6879a14e/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:54:19,567 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=2232a25e9ac1db961952c09e8e81f27b, ASSIGN in 333 msec 2023-03-08 19:54:19,567 INFO [StoreOpener-b4dc975db9245bfec2f0c0dc6879a14e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region b4dc975db9245bfec2f0c0dc6879a14e 2023-03-08 19:54:19,569 DEBUG [StoreOpener-b4dc975db9245bfec2f0c0dc6879a14e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/b4dc975db9245bfec2f0c0dc6879a14e/family2 2023-03-08 19:54:19,569 DEBUG [StoreOpener-b4dc975db9245bfec2f0c0dc6879a14e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/b4dc975db9245bfec2f0c0dc6879a14e/family2 2023-03-08 19:54:19,570 INFO [StoreOpener-b4dc975db9245bfec2f0c0dc6879a14e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b4dc975db9245bfec2f0c0dc6879a14e columnFamilyName family2 2023-03-08 19:54:19,571 INFO [StoreOpener-b4dc975db9245bfec2f0c0dc6879a14e-1] regionserver.HStore(310): Store=b4dc975db9245bfec2f0c0dc6879a14e/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:54:19,571 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/b4dc975db9245bfec2f0c0dc6879a14e 2023-03-08 19:54:19,572 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/b4dc975db9245bfec2f0c0dc6879a14e 2023-03-08 19:54:19,574 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 19:54:19,575 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for b4dc975db9245bfec2f0c0dc6879a14e 2023-03-08 19:54:19,577 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/b4dc975db9245bfec2f0c0dc6879a14e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 19:54:19,577 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b4dc975db9245bfec2f0c0dc6879a14e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75027030, jitterRate=0.11798986792564392}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 19:54:19,577 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b4dc975db9245bfec2f0c0dc6879a14e: 2023-03-08 19:54:19,578 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,,1678305258777.b4dc975db9245bfec2f0c0dc6879a14e., pid=31, masterSystemTime=1678305259542 2023-03-08 19:54:19,580 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,,1678305258777.b4dc975db9245bfec2f0c0dc6879a14e. 2023-03-08 19:54:19,580 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,,1678305258777.b4dc975db9245bfec2f0c0dc6879a14e. 2023-03-08 19:54:19,580 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,2,1678305258777.f43229ba0f524b69e8eb52fe67361236. 2023-03-08 19:54:19,580 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f43229ba0f524b69e8eb52fe67361236, NAME => 'testScannerOnBulkLoadRefHFiles,2,1678305258777.f43229ba0f524b69e8eb52fe67361236.', STARTKEY => '2', ENDKEY => ''} 2023-03-08 19:54:19,580 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=b4dc975db9245bfec2f0c0dc6879a14e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:19,580 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles f43229ba0f524b69e8eb52fe67361236 2023-03-08 19:54:19,580 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,2,1678305258777.f43229ba0f524b69e8eb52fe67361236.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:54:19,580 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,,1678305258777.b4dc975db9245bfec2f0c0dc6879a14e.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678305259580"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678305259580"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678305259580"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678305259580"}]},"ts":"1678305259580"} 2023-03-08 19:54:19,581 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for f43229ba0f524b69e8eb52fe67361236 2023-03-08 19:54:19,581 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for f43229ba0f524b69e8eb52fe67361236 2023-03-08 19:54:19,582 INFO [StoreOpener-f43229ba0f524b69e8eb52fe67361236-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region f43229ba0f524b69e8eb52fe67361236 2023-03-08 19:54:19,583 DEBUG [StoreOpener-f43229ba0f524b69e8eb52fe67361236-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/f43229ba0f524b69e8eb52fe67361236/family1 2023-03-08 19:54:19,583 DEBUG [StoreOpener-f43229ba0f524b69e8eb52fe67361236-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/f43229ba0f524b69e8eb52fe67361236/family1 2023-03-08 19:54:19,584 INFO [StoreOpener-f43229ba0f524b69e8eb52fe67361236-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f43229ba0f524b69e8eb52fe67361236 columnFamilyName family1 2023-03-08 19:54:19,584 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=28 2023-03-08 19:54:19,584 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; OpenRegionProcedure b4dc975db9245bfec2f0c0dc6879a14e, server=jenkins-hbase4.apache.org,33367,1678305218680 in 192 msec 2023-03-08 19:54:19,585 INFO [StoreOpener-f43229ba0f524b69e8eb52fe67361236-1] regionserver.HStore(310): Store=f43229ba0f524b69e8eb52fe67361236/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:54:19,585 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=b4dc975db9245bfec2f0c0dc6879a14e, ASSIGN in 351 msec 2023-03-08 19:54:19,585 INFO [StoreOpener-f43229ba0f524b69e8eb52fe67361236-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region f43229ba0f524b69e8eb52fe67361236 2023-03-08 19:54:19,587 DEBUG [StoreOpener-f43229ba0f524b69e8eb52fe67361236-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/f43229ba0f524b69e8eb52fe67361236/family2 2023-03-08 19:54:19,587 DEBUG [StoreOpener-f43229ba0f524b69e8eb52fe67361236-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/f43229ba0f524b69e8eb52fe67361236/family2 2023-03-08 19:54:19,587 INFO [StoreOpener-f43229ba0f524b69e8eb52fe67361236-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f43229ba0f524b69e8eb52fe67361236 columnFamilyName family2 2023-03-08 19:54:19,588 INFO [StoreOpener-f43229ba0f524b69e8eb52fe67361236-1] regionserver.HStore(310): Store=f43229ba0f524b69e8eb52fe67361236/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:54:19,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/f43229ba0f524b69e8eb52fe67361236 2023-03-08 19:54:19,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/f43229ba0f524b69e8eb52fe67361236 2023-03-08 19:54:19,591 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 19:54:19,592 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for f43229ba0f524b69e8eb52fe67361236 2023-03-08 19:54:19,594 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/f43229ba0f524b69e8eb52fe67361236/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 19:54:19,594 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened f43229ba0f524b69e8eb52fe67361236; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72415806, jitterRate=0.07907959818840027}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 19:54:19,594 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for f43229ba0f524b69e8eb52fe67361236: 2023-03-08 19:54:19,595 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,2,1678305258777.f43229ba0f524b69e8eb52fe67361236., pid=32, masterSystemTime=1678305259542 2023-03-08 19:54:19,596 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,2,1678305258777.f43229ba0f524b69e8eb52fe67361236. 2023-03-08 19:54:19,596 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,2,1678305258777.f43229ba0f524b69e8eb52fe67361236. 2023-03-08 19:54:19,597 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=f43229ba0f524b69e8eb52fe67361236, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:19,597 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,2,1678305258777.f43229ba0f524b69e8eb52fe67361236.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678305259597"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678305259597"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678305259597"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678305259597"}]},"ts":"1678305259597"} 2023-03-08 19:54:19,600 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=30 2023-03-08 19:54:19,600 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=30, state=SUCCESS; OpenRegionProcedure f43229ba0f524b69e8eb52fe67361236, server=jenkins-hbase4.apache.org,33367,1678305218680 in 207 msec 2023-03-08 19:54:19,602 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=27 2023-03-08 19:54:19,602 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=f43229ba0f524b69e8eb52fe67361236, ASSIGN in 367 msec 2023-03-08 19:54:19,603 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 19:54:19,603 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678305259603"}]},"ts":"1678305259603"} 2023-03-08 19:54:19,604 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLED in hbase:meta 2023-03-08 19:54:19,606 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 19:54:19,607 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=27, state=SUCCESS; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles in 828 msec 2023-03-08 19:54:19,888 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-08 19:54:19,888 INFO [Listener at localhost/34049] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testScannerOnBulkLoadRefHFiles, procId: 27 completed 2023-03-08 19:54:19,896 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:19,898 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:19,898 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-03-08 19:54:19,898 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:19,898 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-03-08 19:54:19,898 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-03-08 19:54:19,898 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:19,898 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:19,900 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:19,900 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:19,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:19,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:19,901 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:19,901 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:19,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:19,901 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:19,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:19,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:19,902 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:19,902 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:19,902 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-03-08 19:54:19,902 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:19,902 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-03-08 19:54:19,902 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1678305258777.2232a25e9ac1db961952c09e8e81f27b. 2023-03-08 19:54:19,902 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1678305258777.2232a25e9ac1db961952c09e8e81f27b. started... 2023-03-08 19:54:19,902 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1678305258777.b4dc975db9245bfec2f0c0dc6879a14e. 2023-03-08 19:54:19,902 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1678305258777.b4dc975db9245bfec2f0c0dc6879a14e. started... 2023-03-08 19:54:19,902 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 19:54:19,903 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2232a25e9ac1db961952c09e8e81f27b: 2023-03-08 19:54:19,903 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:19,903 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for b4dc975db9245bfec2f0c0dc6879a14e: 2023-03-08 19:54:19,903 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1678305258777.f43229ba0f524b69e8eb52fe67361236. 2023-03-08 19:54:19,903 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1678305258777.b4dc975db9245bfec2f0c0dc6879a14e. 2023-03-08 19:54:19,903 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1678305258777.2232a25e9ac1db961952c09e8e81f27b. 2023-03-08 19:54:19,903 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 19:54:19,903 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1678305258777.f43229ba0f524b69e8eb52fe67361236. started... 2023-03-08 19:54:19,903 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 19:54:19,904 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing f43229ba0f524b69e8eb52fe67361236 2/2 column families, dataSize=48 B heapSize=632 B 2023-03-08 19:54:20,325 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308ea06fc43dfd6470f8b31b532c9e75195 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230308ea06fc43dfd6470f8b31b532c9e75195 2023-03-08 19:54:20,731 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=48, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/f43229ba0f524b69e8eb52fe67361236/.tmp/family1/fc1e73fdf16944fc901a8f281afd51c6 2023-03-08 19:54:20,736 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/f43229ba0f524b69e8eb52fe67361236/.tmp/family1/fc1e73fdf16944fc901a8f281afd51c6 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/f43229ba0f524b69e8eb52fe67361236/family1/fc1e73fdf16944fc901a8f281afd51c6 2023-03-08 19:54:20,741 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/f43229ba0f524b69e8eb52fe67361236/family1/fc1e73fdf16944fc901a8f281afd51c6, entries=1, sequenceid=5, filesize=4.9 K 2023-03-08 19:54:20,742 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~48 B/48, heapSize ~360 B/360, currentSize=0 B/0 for f43229ba0f524b69e8eb52fe67361236 in 838ms, sequenceid=5, compaction requested=false 2023-03-08 19:54:20,743 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testScannerOnBulkLoadRefHFiles' 2023-03-08 19:54:20,743 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for f43229ba0f524b69e8eb52fe67361236: 2023-03-08 19:54:20,743 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1678305258777.f43229ba0f524b69e8eb52fe67361236. 2023-03-08 19:54:20,743 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 19:54:20,743 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 19:54:20,743 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:20,744 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-03-08 19:54:20,744 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-03-08 19:54:20,746 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:20,746 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:20,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:20,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:20,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:20,747 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:20,747 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:20,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:20,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:20,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:20,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:20,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:20,748 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-03-08 19:54:20,748 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6522df96[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:20,748 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-03-08 19:54:20,748 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:20,750 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:20,750 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:20,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:20,750 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-03-08 19:54:20,750 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-03-08 19:54:20,750 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:20,750 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:20,750 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:20,752 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-03-08 19:54:20,752 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:20,752 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:20,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:20,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:20,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:20,752 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-03-08 19:54:20,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:20,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:20,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:20,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:20,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:20,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:20,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:20,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:20,755 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-03-08 19:54:20,755 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-03-08 19:54:20,755 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:20,755 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:20,755 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:20,755 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:20,757 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:20,757 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:20,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:20,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:20,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:20,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:20,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:20,757 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:20,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:20,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:20,758 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:20,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:20,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:20,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:20,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:20,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:20,759 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:20,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:20,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:20,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:20,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:20,764 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:20,764 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:20,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:20,764 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-03-08 19:54:20,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:20,764 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-03-08 19:54:20,764 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:20,764 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:20,764 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:20,764 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:20,764 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:20,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:20,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:20,765 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:20,765 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:20,765 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:20,765 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:20,765 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:20,765 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:20,865 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:20,866 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:20,870 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:20,872 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testScannerOnBulkLoadRefHFiles was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:20,872 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:20,872 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-03-08 19:54:20,873 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:20,873 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-03-08 19:54:20,873 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-03-08 19:54:20,873 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:20,873 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:20,875 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:20,875 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:20,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:20,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:20,875 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:20,875 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:20,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:20,875 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:20,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:20,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:20,876 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:20,876 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testScannerOnBulkLoadRefHFiles is still present, removing 2023-03-08 19:54:20,876 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:20,876 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-03-08 19:54:20,876 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:20,876 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-03-08 19:54:20,876 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1678305258777.2232a25e9ac1db961952c09e8e81f27b. 2023-03-08 19:54:20,876 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1678305258777.2232a25e9ac1db961952c09e8e81f27b. started... 2023-03-08 19:54:20,876 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1678305258777.b4dc975db9245bfec2f0c0dc6879a14e. 2023-03-08 19:54:20,876 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1678305258777.b4dc975db9245bfec2f0c0dc6879a14e. started... 2023-03-08 19:54:20,876 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1678305258777.f43229ba0f524b69e8eb52fe67361236. 2023-03-08 19:54:20,876 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 19:54:20,877 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for b4dc975db9245bfec2f0c0dc6879a14e: 2023-03-08 19:54:20,877 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:20,877 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1678305258777.f43229ba0f524b69e8eb52fe67361236. started... 2023-03-08 19:54:20,877 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2232a25e9ac1db961952c09e8e81f27b: 2023-03-08 19:54:20,877 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1678305258777.2232a25e9ac1db961952c09e8e81f27b. 2023-03-08 19:54:20,877 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1678305258777.b4dc975db9245bfec2f0c0dc6879a14e. 2023-03-08 19:54:20,877 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 19:54:20,877 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 19:54:20,877 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing f43229ba0f524b69e8eb52fe67361236 2/2 column families, dataSize=90 B heapSize=752 B 2023-03-08 19:54:21,291 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023030800b1e04ef7fa460296b95764ac3dd169 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023030800b1e04ef7fa460296b95764ac3dd169 2023-03-08 19:54:21,696 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=90, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/f43229ba0f524b69e8eb52fe67361236/.tmp/family1/7cac6a3d6f5545218f888330a6512653 2023-03-08 19:54:21,701 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/f43229ba0f524b69e8eb52fe67361236/.tmp/family1/7cac6a3d6f5545218f888330a6512653 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/f43229ba0f524b69e8eb52fe67361236/family1/7cac6a3d6f5545218f888330a6512653 2023-03-08 19:54:21,706 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/f43229ba0f524b69e8eb52fe67361236/family1/7cac6a3d6f5545218f888330a6512653, entries=2, sequenceid=9, filesize=5.0 K 2023-03-08 19:54:21,706 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~90 B/90, heapSize ~480 B/480, currentSize=0 B/0 for f43229ba0f524b69e8eb52fe67361236 in 829ms, sequenceid=9, compaction requested=false 2023-03-08 19:54:21,707 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for f43229ba0f524b69e8eb52fe67361236: 2023-03-08 19:54:21,707 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1678305258777.f43229ba0f524b69e8eb52fe67361236. 2023-03-08 19:54:21,707 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 19:54:21,707 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 19:54:21,707 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:21,708 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-03-08 19:54:21,708 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-03-08 19:54:21,710 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:21,710 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:21,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:21,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:21,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:21,710 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:21,711 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:21,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:21,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:21,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:21,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:21,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:21,712 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-03-08 19:54:21,712 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@52276dc9[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:21,712 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-03-08 19:54:21,712 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:21,713 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:21,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:21,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:21,714 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-03-08 19:54:21,714 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-03-08 19:54:21,714 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:21,714 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:21,714 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:21,715 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:21,715 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-03-08 19:54:21,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:21,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:21,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:21,715 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:21,715 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-03-08 19:54:21,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:21,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:21,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:21,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:21,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:21,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:21,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:21,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:21,717 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-03-08 19:54:21,717 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-03-08 19:54:21,717 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:21,717 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:21,717 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:21,717 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:21,719 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:21,719 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:21,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:21,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:21,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:21,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:21,719 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:21,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:21,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:21,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:21,720 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:21,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:21,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:21,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:21,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:21,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:21,721 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:21,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:21,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:21,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:21,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:21,725 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:21,725 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:21,725 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:21,725 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:21,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:21,725 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:21,725 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:21,725 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-03-08 19:54:21,725 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:21,725 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:21,725 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:21,725 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:21,725 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:21,725 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:21,725 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:21,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:21,725 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:21,726 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:21,826 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:21,827 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:21,834 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39989] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testScannerOnBulkLoadRefHFiles 2023-03-08 19:54:21,836 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] procedure2.ProcedureExecutor(1029): Stored pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-03-08 19:54:21,837 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-03-08 19:54:21,838 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] mob.MobUtils(833): start MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 19:54:21,838 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=34, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678305861838 2023-03-08 19:54:21,839 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 19:54:21,839 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testScannerOnBulkLoadRefHFiles' and column 'family1' 2023-03-08 19:54:21,839 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-08 19:54:21,840 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] zookeeper.ReadOnlyZKClient(139): Connect 0x6dec2b43 to 127.0.0.1:56751 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 19:54:21,844 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5c660a86, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 19:54:21,844 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230308 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-03-08 19:54:22,676 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303088fb4d61067bd4adfb33cb87247b99262 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202303088fb4d61067bd4adfb33cb87247b99262 2023-03-08 19:54:22,677 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230308 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-08 19:54:22,690 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 19:54:22,691 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60296, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 19:54:22,699 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 19:54:22,701 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58672, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 19:54:22,710 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230308/family1/7b36640863e842e08b16fe2a00c1f867 first=Optional[k0] last=Optional[k1] 2023-03-08 19:54:22,711 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testScannerOnBulkLoadRefHFiles,2,1678305258777.f43229ba0f524b69e8eb52fe67361236., hostname=jenkins-hbase4.apache.org,33367,1678305218680, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230308/family1/7b36640863e842e08b16fe2a00c1f867}] 2023-03-08 19:54:22,716 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(763): Validating hfile at hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230308/family1/7b36640863e842e08b16fe2a00c1f867 for inclusion in f43229ba0f524b69e8eb52fe67361236/family1 2023-03-08 19:54:22,718 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(775): HFile bounds: first=k0 last=k1 2023-03-08 19:54:22,718 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-08 19:54:22,719 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(2446): Flush status journal for f43229ba0f524b69e8eb52fe67361236: 2023-03-08 19:54:22,719 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230308/family1/7b36640863e842e08b16fe2a00c1f867 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testScannerOnBulkLoadRefHFiles__p766shu6tjtf5mmt9o9gcpoqudvdgtuarqms0g9ai0s55dukgijub6gmm5pv35mp/family1/7b36640863e842e08b16fe2a00c1f867 2023-03-08 19:54:22,722 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testScannerOnBulkLoadRefHFiles__p766shu6tjtf5mmt9o9gcpoqudvdgtuarqms0g9ai0s55dukgijub6gmm5pv35mp/family1/7b36640863e842e08b16fe2a00c1f867 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/f43229ba0f524b69e8eb52fe67361236/family1/01ad0b254e3d41fb81825bf71bce2862_SeqId_12_ 2023-03-08 19:54:22,722 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(849): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testScannerOnBulkLoadRefHFiles__p766shu6tjtf5mmt9o9gcpoqudvdgtuarqms0g9ai0s55dukgijub6gmm5pv35mp/family1/7b36640863e842e08b16fe2a00c1f867 into f43229ba0f524b69e8eb52fe67361236/family1 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/f43229ba0f524b69e8eb52fe67361236/family1/01ad0b254e3d41fb81825bf71bce2862_SeqId_12_ - updating store file list. 2023-03-08 19:54:22,726 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(882): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/f43229ba0f524b69e8eb52fe67361236/family1/01ad0b254e3d41fb81825bf71bce2862_SeqId_12_ into f43229ba0f524b69e8eb52fe67361236/family1 2023-03-08 19:54:22,726 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(855): Successfully loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testScannerOnBulkLoadRefHFiles__p766shu6tjtf5mmt9o9gcpoqudvdgtuarqms0g9ai0s55dukgijub6gmm5pv35mp/family1/7b36640863e842e08b16fe2a00c1f867 into f43229ba0f524b69e8eb52fe67361236/family1 (new location: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/f43229ba0f524b69e8eb52fe67361236/family1/01ad0b254e3d41fb81825bf71bce2862_SeqId_12_) 2023-03-08 19:54:22,726 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testScannerOnBulkLoadRefHFiles__p766shu6tjtf5mmt9o9gcpoqudvdgtuarqms0g9ai0s55dukgijub6gmm5pv35mp/family1/7b36640863e842e08b16fe2a00c1f867 2023-03-08 19:54:22,726 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-08 19:54:22,727 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 15445 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-08 19:54:22,727 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(1912): f43229ba0f524b69e8eb52fe67361236/family1 is initiating minor compaction (all files) 2023-03-08 19:54:22,727 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@42bf60c1; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 19:54:22,727 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of f43229ba0f524b69e8eb52fe67361236/family1 in testScannerOnBulkLoadRefHFiles,2,1678305258777.f43229ba0f524b69e8eb52fe67361236. 2023-03-08 19:54:22,728 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/f43229ba0f524b69e8eb52fe67361236/family1/fc1e73fdf16944fc901a8f281afd51c6, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/f43229ba0f524b69e8eb52fe67361236/family1/7cac6a3d6f5545218f888330a6512653, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/f43229ba0f524b69e8eb52fe67361236/family1/01ad0b254e3d41fb81825bf71bce2862_SeqId_12_] into tmpdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/f43229ba0f524b69e8eb52fe67361236/.tmp, totalSize=15.1 K 2023-03-08 19:54:22,728 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting fc1e73fdf16944fc901a8f281afd51c6, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678305259888 2023-03-08 19:54:22,728 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 7cac6a3d6f5545218f888330a6512653, keycount=2, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678305259888 2023-03-08 19:54:22,729 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 01ad0b254e3d41fb81825bf71bce2862_SeqId_12_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678305259888 2023-03-08 19:54:22,729 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230308 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-08 19:54:22,730 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:54:22,732 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023030800b1e04ef7fa460296b95764ac3dd169 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023030800b1e04ef7fa460296b95764ac3dd169 2023-03-08 19:54:22,733 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230308ea06fc43dfd6470f8b31b532c9e75195 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230308ea06fc43dfd6470f8b31b532c9e75195 2023-03-08 19:54:22,733 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testScannerOnBulkLoadRefHFiles 2 expired mob files are deleted 2023-03-08 19:54:22,733 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230308' 2023-03-08 19:54:22,734 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230308 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-03-08 19:54:22,734 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-08 19:54:22,734 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] mob.MobUtils(840): end MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 19:54:22,735 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=34, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-03-08 19:54:22,737 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=34, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE in 900 msec 2023-03-08 19:54:22,742 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] throttle.PressureAwareThroughputController(145): f43229ba0f524b69e8eb52fe67361236#family1#compaction#38 average throughput is 0.03 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 19:54:22,776 INFO [Listener at localhost/34049] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=255 (was 257), OpenFileDescriptor=610 (was 603) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=233 (was 219) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=8280 (was 8484) 2023-03-08 19:54:22,790 INFO [Listener at localhost/34049] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=255, OpenFileDescriptor=610, MaxFileDescriptor=60000, SystemLoadAverage=233, ProcessCount=170, AvailableMemoryMB=8280 2023-03-08 19:54:22,792 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-08 19:54:22,793 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:22,794 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 19:54:22,795 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyPolicy" procId is: 35 2023-03-08 19:54:22,795 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 19:54:22,795 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-08 19:54:22,796 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764 2023-03-08 19:54:22,797 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764 empty. 2023-03-08 19:54:22,798 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764 2023-03-08 19:54:22,798 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyPolicy regions 2023-03-08 19:54:22,897 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-08 19:54:23,098 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-08 19:54:23,206 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-08 19:54:23,207 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => 2d54b0ef5ff21a425fa7a6fc1850c764, NAME => 'testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp 2023-03-08 19:54:23,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-08 19:54:23,613 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:54:23,614 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1604): Closing 2d54b0ef5ff21a425fa7a6fc1850c764, disabling compactions & flushes 2023-03-08 19:54:23,614 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. 2023-03-08 19:54:23,614 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. 2023-03-08 19:54:23,614 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. after waiting 0 ms 2023-03-08 19:54:23,614 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. 2023-03-08 19:54:23,614 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. 2023-03-08 19:54:23,614 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for 2d54b0ef5ff21a425fa7a6fc1850c764: 2023-03-08 19:54:23,616 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 19:54:23,617 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678305263617"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678305263617"}]},"ts":"1678305263617"} 2023-03-08 19:54:23,619 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-08 19:54:23,619 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 19:54:23,619 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678305263619"}]},"ts":"1678305263619"} 2023-03-08 19:54:23,620 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-03-08 19:54:23,624 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=2d54b0ef5ff21a425fa7a6fc1850c764, ASSIGN}] 2023-03-08 19:54:23,626 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=2d54b0ef5ff21a425fa7a6fc1850c764, ASSIGN 2023-03-08 19:54:23,626 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=2d54b0ef5ff21a425fa7a6fc1850c764, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33367,1678305218680; forceNewPlan=false, retain=false 2023-03-08 19:54:23,777 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=2d54b0ef5ff21a425fa7a6fc1850c764, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:23,777 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678305263777"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678305263777"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678305263777"}]},"ts":"1678305263777"} 2023-03-08 19:54:23,779 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=37, ppid=36, state=RUNNABLE; OpenRegionProcedure 2d54b0ef5ff21a425fa7a6fc1850c764, server=jenkins-hbase4.apache.org,33367,1678305218680}] 2023-03-08 19:54:23,900 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-08 19:54:23,934 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. 2023-03-08 19:54:23,934 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2d54b0ef5ff21a425fa7a6fc1850c764, NAME => 'testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764.', STARTKEY => '', ENDKEY => ''} 2023-03-08 19:54:23,935 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyPolicy 2d54b0ef5ff21a425fa7a6fc1850c764 2023-03-08 19:54:23,935 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:54:23,935 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2d54b0ef5ff21a425fa7a6fc1850c764 2023-03-08 19:54:23,935 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2d54b0ef5ff21a425fa7a6fc1850c764 2023-03-08 19:54:23,936 INFO [StoreOpener-2d54b0ef5ff21a425fa7a6fc1850c764-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 2d54b0ef5ff21a425fa7a6fc1850c764 2023-03-08 19:54:23,937 DEBUG [StoreOpener-2d54b0ef5ff21a425fa7a6fc1850c764-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1 2023-03-08 19:54:23,937 DEBUG [StoreOpener-2d54b0ef5ff21a425fa7a6fc1850c764-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1 2023-03-08 19:54:23,938 INFO [StoreOpener-2d54b0ef5ff21a425fa7a6fc1850c764-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2d54b0ef5ff21a425fa7a6fc1850c764 columnFamilyName family1 2023-03-08 19:54:23,939 INFO [StoreOpener-2d54b0ef5ff21a425fa7a6fc1850c764-1] regionserver.HStore(310): Store=2d54b0ef5ff21a425fa7a6fc1850c764/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:54:23,940 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764 2023-03-08 19:54:23,940 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764 2023-03-08 19:54:23,943 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2d54b0ef5ff21a425fa7a6fc1850c764 2023-03-08 19:54:23,945 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 19:54:23,946 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2d54b0ef5ff21a425fa7a6fc1850c764; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72747110, jitterRate=0.08401641249656677}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 19:54:23,946 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2d54b0ef5ff21a425fa7a6fc1850c764: 2023-03-08 19:54:23,946 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764., pid=37, masterSystemTime=1678305263931 2023-03-08 19:54:23,948 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. 2023-03-08 19:54:23,948 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. 2023-03-08 19:54:23,949 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=2d54b0ef5ff21a425fa7a6fc1850c764, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:23,949 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678305263948"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678305263948"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678305263948"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678305263948"}]},"ts":"1678305263948"} 2023-03-08 19:54:23,952 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=36 2023-03-08 19:54:23,952 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=36, state=SUCCESS; OpenRegionProcedure 2d54b0ef5ff21a425fa7a6fc1850c764, server=jenkins-hbase4.apache.org,33367,1678305218680 in 171 msec 2023-03-08 19:54:23,954 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-03-08 19:54:23,954 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=2d54b0ef5ff21a425fa7a6fc1850c764, ASSIGN in 328 msec 2023-03-08 19:54:23,954 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 19:54:23,954 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678305263954"}]},"ts":"1678305263954"} 2023-03-08 19:54:23,955 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-03-08 19:54:23,958 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 19:54:23,960 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy in 1.1670 sec 2023-03-08 19:54:23,963 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/f43229ba0f524b69e8eb52fe67361236/.tmp/family1/ad967154f5fa4b7484c85c20f47c5902 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/f43229ba0f524b69e8eb52fe67361236/family1/ad967154f5fa4b7484c85c20f47c5902 2023-03-08 19:54:23,969 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in f43229ba0f524b69e8eb52fe67361236/family1 of f43229ba0f524b69e8eb52fe67361236 into ad967154f5fa4b7484c85c20f47c5902(size=5.1 K), total size for store is 5.1 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 19:54:23,969 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for f43229ba0f524b69e8eb52fe67361236: 2023-03-08 19:54:23,969 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testScannerOnBulkLoadRefHFiles,2,1678305258777.f43229ba0f524b69e8eb52fe67361236., storeName=family1, fileCount=3, fileSize=15.1 K (4.9 K, 5.0 K, 5.2 K), priority=2, time=1678305262727; duration=1sec 2023-03-08 19:54:23,969 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 19:54:24,016 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/f43229ba0f524b69e8eb52fe67361236/family1/fc1e73fdf16944fc901a8f281afd51c6, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/f43229ba0f524b69e8eb52fe67361236/family1/7cac6a3d6f5545218f888330a6512653, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/f43229ba0f524b69e8eb52fe67361236/family1/01ad0b254e3d41fb81825bf71bce2862_SeqId_12_] to archive 2023-03-08 19:54:24,018 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:54:24,020 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/f43229ba0f524b69e8eb52fe67361236/family1/fc1e73fdf16944fc901a8f281afd51c6 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testScannerOnBulkLoadRefHFiles/f43229ba0f524b69e8eb52fe67361236/family1/fc1e73fdf16944fc901a8f281afd51c6 2023-03-08 19:54:24,021 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/f43229ba0f524b69e8eb52fe67361236/family1/7cac6a3d6f5545218f888330a6512653 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testScannerOnBulkLoadRefHFiles/f43229ba0f524b69e8eb52fe67361236/family1/7cac6a3d6f5545218f888330a6512653 2023-03-08 19:54:24,023 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/f43229ba0f524b69e8eb52fe67361236/family1/01ad0b254e3d41fb81825bf71bce2862_SeqId_12_ to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testScannerOnBulkLoadRefHFiles/f43229ba0f524b69e8eb52fe67361236/family1/01ad0b254e3d41fb81825bf71bce2862_SeqId_12_ 2023-03-08 19:54:24,902 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-08 19:54:24,902 INFO [Listener at localhost/34049] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyPolicy, procId: 35 completed 2023-03-08 19:54:24,923 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:24,925 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:24,925 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 19:54:24,925 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:24,925 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 19:54:24,925 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 19:54:24,926 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:24,926 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:24,927 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:24,927 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:24,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:24,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:24,927 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:24,927 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:24,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:24,928 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:24,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:24,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:24,928 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:24,928 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:24,928 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 19:54:24,928 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:24,928 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 19:54:24,929 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:24,929 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:24,929 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. 2023-03-08 19:54:24,929 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. started... 2023-03-08 19:54:24,929 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2d54b0ef5ff21a425fa7a6fc1850c764 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-08 19:54:25,048 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 19:54:25,352 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113009bf59d4ebd44cdb96d6d218cea3fd74 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015113009bf59d4ebd44cdb96d6d218cea3fd74 2023-03-08 19:54:25,756 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/.tmp/family1/a9ac14e4aba34b548aad689d0dc09957 2023-03-08 19:54:25,761 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/.tmp/family1/a9ac14e4aba34b548aad689d0dc09957 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/a9ac14e4aba34b548aad689d0dc09957 2023-03-08 19:54:25,765 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/a9ac14e4aba34b548aad689d0dc09957, entries=1000, sequenceid=5, filesize=165.1 K 2023-03-08 19:54:25,766 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 2d54b0ef5ff21a425fa7a6fc1850c764 in 837ms, sequenceid=5, compaction requested=false 2023-03-08 19:54:25,766 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 19:54:25,766 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2d54b0ef5ff21a425fa7a6fc1850c764: 2023-03-08 19:54:25,766 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. 2023-03-08 19:54:25,766 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:25,766 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:25,766 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:25,766 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 19:54:25,766 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-08 19:54:25,769 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:25,769 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:25,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:25,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:25,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:25,769 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:25,769 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:25,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:25,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:25,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:25,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:25,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:25,770 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 19:54:25,770 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@11568ec4[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:25,770 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 19:54:25,770 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:25,772 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:25,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:25,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:25,772 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 19:54:25,772 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-08 19:54:25,772 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:25,772 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:25,772 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:25,774 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 19:54:25,774 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:25,774 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:25,774 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-08 19:54:25,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:25,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:25,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:25,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:25,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:25,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:25,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:25,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:25,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:25,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:25,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:25,776 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 19:54:25,776 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-08 19:54:25,777 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:25,777 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:25,777 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:25,777 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:25,778 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:25,778 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:25,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:25,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:25,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:25,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:25,778 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:25,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:25,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:25,779 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:25,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:25,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:25,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:25,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:25,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:25,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:25,780 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:25,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:25,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:25,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:25,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:25,782 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:25,782 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:25,782 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:25,783 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:25,783 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:25,783 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:25,783 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:25,783 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 19:54:25,783 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:25,783 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:25,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:25,783 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 19:54:25,783 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:25,783 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:25,783 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:25,783 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:25,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:25,783 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:25,783 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:25,883 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:25,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:25,889 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:25,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:25,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:25,890 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 19:54:25,890 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:25,890 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 19:54:25,890 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 19:54:25,891 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:25,891 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:25,892 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:25,892 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:25,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:25,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:25,892 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:25,892 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:25,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:25,893 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:25,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:25,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:25,893 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:25,894 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 19:54:25,894 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:25,894 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 19:54:25,894 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:25,894 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 19:54:25,894 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:25,894 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:25,894 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. 2023-03-08 19:54:25,894 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. started... 2023-03-08 19:54:25,894 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2d54b0ef5ff21a425fa7a6fc1850c764 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-08 19:54:26,307 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112803b7ed451e654ca889b912945d5bfdb3 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015112803b7ed451e654ca889b912945d5bfdb3 2023-03-08 19:54:26,714 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/.tmp/family1/aedba52f549e45ae85769a5c7a88b987 2023-03-08 19:54:26,720 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/.tmp/family1/aedba52f549e45ae85769a5c7a88b987 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/aedba52f549e45ae85769a5c7a88b987 2023-03-08 19:54:26,724 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/aedba52f549e45ae85769a5c7a88b987, entries=1, sequenceid=9, filesize=5.0 K 2023-03-08 19:54:26,725 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 2d54b0ef5ff21a425fa7a6fc1850c764 in 831ms, sequenceid=9, compaction requested=false 2023-03-08 19:54:26,725 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2d54b0ef5ff21a425fa7a6fc1850c764: 2023-03-08 19:54:26,725 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. 2023-03-08 19:54:26,726 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:26,726 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:26,726 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:26,726 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 19:54:26,726 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-08 19:54:26,727 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:26,727 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:26,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:26,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:26,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:26,728 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:26,728 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:26,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:26,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:26,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:26,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:26,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:26,729 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 19:54:26,729 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6abbdd6e[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:26,729 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 19:54:26,729 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:26,731 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:26,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:26,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:26,731 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 19:54:26,731 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-08 19:54:26,731 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:26,731 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:26,731 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:26,737 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:26,737 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 19:54:26,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:26,737 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:26,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:26,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:26,737 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-08 19:54:26,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:26,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:26,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:26,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:26,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:26,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:26,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:26,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:26,739 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 19:54:26,739 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-08 19:54:26,739 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:26,739 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:26,739 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:26,739 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:26,741 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:26,741 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:26,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:26,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:26,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:26,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:26,741 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:26,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:26,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:26,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:26,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:26,742 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:26,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:26,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:26,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:26,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:26,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:26,743 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:26,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:26,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:26,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:26,745 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:26,745 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:26,745 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:26,745 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:26,745 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:26,745 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:26,745 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 19:54:26,746 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:26,745 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:26,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:26,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:26,746 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:26,746 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:26,746 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:26,746 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:26,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:26,746 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:26,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:26,846 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:26,847 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:26,851 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:26,853 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:26,853 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:26,853 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 19:54:26,853 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:26,853 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 19:54:26,853 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 19:54:26,853 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:26,853 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:26,855 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:26,855 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:26,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:26,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:26,855 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:26,855 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:26,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:26,855 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:26,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:26,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:26,856 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:26,856 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 19:54:26,856 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:26,856 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 19:54:26,856 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:26,856 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 19:54:26,856 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:26,856 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:26,856 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. 2023-03-08 19:54:26,857 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. started... 2023-03-08 19:54:26,857 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2d54b0ef5ff21a425fa7a6fc1850c764 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:54:27,270 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512010c6a785f419c4d1ca49c97abafb14d48 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512010c6a785f419c4d1ca49c97abafb14d48 2023-03-08 19:54:27,676 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/.tmp/family1/9fe74ddadac843ccb5845ca688f25c9c 2023-03-08 19:54:27,681 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/.tmp/family1/9fe74ddadac843ccb5845ca688f25c9c as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/9fe74ddadac843ccb5845ca688f25c9c 2023-03-08 19:54:27,684 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/9fe74ddadac843ccb5845ca688f25c9c, entries=1, sequenceid=13, filesize=4.9 K 2023-03-08 19:54:27,685 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2d54b0ef5ff21a425fa7a6fc1850c764 in 828ms, sequenceid=13, compaction requested=true 2023-03-08 19:54:27,685 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2d54b0ef5ff21a425fa7a6fc1850c764: 2023-03-08 19:54:27,685 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. 2023-03-08 19:54:27,685 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:27,685 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:27,685 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:27,686 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 19:54:27,686 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-08 19:54:27,688 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:27,688 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:27,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:27,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:27,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:27,689 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:27,689 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:27,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:27,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:27,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:27,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:27,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:27,690 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 19:54:27,690 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@191661c0[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:27,690 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 19:54:27,690 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:27,691 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:27,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:27,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:27,691 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 19:54:27,691 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-08 19:54:27,691 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:27,691 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:27,692 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:27,693 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 19:54:27,693 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:27,693 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:27,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:27,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:27,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:27,693 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-08 19:54:27,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:27,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:27,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:27,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:27,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:27,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:27,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:27,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:27,695 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 19:54:27,695 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-08 19:54:27,695 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:27,695 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:27,695 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:27,695 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:27,697 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:27,697 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:27,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:27,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:27,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:27,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:27,697 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:27,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:27,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:27,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:27,698 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:27,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:27,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:27,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:27,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:27,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:27,699 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:27,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:27,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:27,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:27,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:27,702 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:27,703 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:27,703 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:27,703 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:27,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:27,703 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:27,703 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:27,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:27,703 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 19:54:27,703 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:27,703 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:27,703 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:27,703 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:27,703 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:27,703 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 19:54:27,703 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:27,704 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:27,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:27,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:27,804 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:27,805 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:27,809 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:27,810 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:27,811 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:27,811 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 19:54:27,811 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:27,811 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 19:54:27,811 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 19:54:27,811 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:27,811 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:27,812 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:27,812 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:27,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:27,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:27,813 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:27,813 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:27,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:27,813 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:27,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:27,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:27,814 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:27,814 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 19:54:27,814 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:27,814 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 19:54:27,814 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:27,814 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 19:54:27,814 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:27,814 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:27,814 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. 2023-03-08 19:54:27,814 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. started... 2023-03-08 19:54:27,814 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2d54b0ef5ff21a425fa7a6fc1850c764 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:54:27,827 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512059480d3f4139d452aa7c93f5821e834f6 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512059480d3f4139d452aa7c93f5821e834f6 2023-03-08 19:54:28,234 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/.tmp/family1/de38a040076048ceb34ac6d5eabe452e 2023-03-08 19:54:28,239 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/.tmp/family1/de38a040076048ceb34ac6d5eabe452e as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/de38a040076048ceb34ac6d5eabe452e 2023-03-08 19:54:28,243 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/de38a040076048ceb34ac6d5eabe452e, entries=1, sequenceid=17, filesize=4.9 K 2023-03-08 19:54:28,244 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2d54b0ef5ff21a425fa7a6fc1850c764 in 430ms, sequenceid=17, compaction requested=true 2023-03-08 19:54:28,244 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2d54b0ef5ff21a425fa7a6fc1850c764: 2023-03-08 19:54:28,245 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. 2023-03-08 19:54:28,245 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:28,245 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:28,245 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:28,245 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 19:54:28,245 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-08 19:54:28,246 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:28,247 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:28,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:28,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:28,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:28,247 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:28,247 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:28,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:28,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:28,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:28,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:28,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:28,248 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 19:54:28,248 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@70360e0e[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:28,248 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 19:54:28,248 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:28,249 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:28,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:28,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:28,250 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 19:54:28,250 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-08 19:54:28,250 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:28,250 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:28,250 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:28,252 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 19:54:28,252 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:28,252 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:28,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:28,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:28,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:28,252 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-08 19:54:28,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:28,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:28,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:28,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:28,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:28,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:28,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:28,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:28,254 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 19:54:28,254 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-08 19:54:28,254 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:28,254 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:28,254 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:28,254 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:28,256 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:28,256 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:28,256 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:28,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:28,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:28,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:28,256 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:28,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:28,256 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:28,256 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:28,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:28,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:28,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:28,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:28,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:28,257 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:28,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:28,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:28,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:28,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:28,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:28,259 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:28,259 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:28,259 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:28,259 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:28,260 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:28,260 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:28,260 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 19:54:28,260 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:28,260 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:28,260 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:28,260 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 19:54:28,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:28,260 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:28,260 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:28,260 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:28,260 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:28,260 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:28,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:28,260 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:28,360 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:28,361 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:28,366 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:28,367 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:28,367 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:28,367 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 19:54:28,367 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:28,367 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 19:54:28,368 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 19:54:28,368 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:28,368 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:28,369 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:28,369 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:28,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:28,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:28,370 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:28,370 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:28,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:28,370 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:28,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:28,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:28,371 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:28,371 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 19:54:28,371 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:28,371 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 19:54:28,371 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:28,371 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 19:54:28,371 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:28,371 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:28,371 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. 2023-03-08 19:54:28,371 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. started... 2023-03-08 19:54:28,371 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2d54b0ef5ff21a425fa7a6fc1850c764 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:54:28,785 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228dc5a9cedef8549f8ab3d2081a8ac79f2 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151228dc5a9cedef8549f8ab3d2081a8ac79f2 2023-03-08 19:54:29,191 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/.tmp/family1/3c979f53dcbd40eaa6ecdaa6ec42421b 2023-03-08 19:54:29,196 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/.tmp/family1/3c979f53dcbd40eaa6ecdaa6ec42421b as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/3c979f53dcbd40eaa6ecdaa6ec42421b 2023-03-08 19:54:29,200 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/3c979f53dcbd40eaa6ecdaa6ec42421b, entries=1, sequenceid=21, filesize=4.9 K 2023-03-08 19:54:29,201 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2d54b0ef5ff21a425fa7a6fc1850c764 in 830ms, sequenceid=21, compaction requested=true 2023-03-08 19:54:29,201 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2d54b0ef5ff21a425fa7a6fc1850c764: 2023-03-08 19:54:29,201 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. 2023-03-08 19:54:29,201 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:29,201 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:29,201 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:29,201 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 19:54:29,202 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-08 19:54:29,211 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:29,211 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:29,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:29,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:29,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:29,212 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:29,212 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:29,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:29,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:29,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:29,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:29,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:29,213 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 19:54:29,213 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@55338e62[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:29,213 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 19:54:29,213 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:29,214 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:29,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:29,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:29,215 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 19:54:29,215 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:29,215 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:29,215 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-08 19:54:29,215 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:29,216 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:29,216 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 19:54:29,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:29,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:29,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:29,216 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:29,216 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-08 19:54:29,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:29,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:29,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:29,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:29,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:29,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:29,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:29,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:29,218 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 19:54:29,218 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-08 19:54:29,219 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:29,219 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:29,219 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:29,219 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:29,220 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:29,220 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:29,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:29,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:29,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:29,220 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:29,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:29,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:29,221 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:29,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:29,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:29,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:29,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:29,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:29,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:29,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:29,222 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:29,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:29,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:29,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:29,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:29,225 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:29,225 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:29,225 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:29,225 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:29,225 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 19:54:29,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:29,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:29,225 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 19:54:29,226 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:29,225 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:29,225 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:29,225 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:29,226 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:29,226 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:29,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:29,226 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:29,226 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:29,226 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:29,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:29,326 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:29,327 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:29,331 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:29,333 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:29,333 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:29,333 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 19:54:29,333 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:29,333 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 19:54:29,333 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 19:54:29,333 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:29,333 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:29,335 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:29,335 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:29,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:29,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:29,335 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:29,335 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:29,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:29,336 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:29,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:29,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:29,336 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:29,336 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 19:54:29,336 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:29,336 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 19:54:29,336 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:29,336 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 19:54:29,336 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:29,336 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:29,337 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. 2023-03-08 19:54:29,337 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. started... 2023-03-08 19:54:29,337 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2d54b0ef5ff21a425fa7a6fc1850c764 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:54:29,749 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231afe314b5fc974827b1725a58a891e5bd to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231afe314b5fc974827b1725a58a891e5bd 2023-03-08 19:54:30,155 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/.tmp/family1/7a0dfa548faa42c19d07fe7883244b92 2023-03-08 19:54:30,160 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/.tmp/family1/7a0dfa548faa42c19d07fe7883244b92 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/7a0dfa548faa42c19d07fe7883244b92 2023-03-08 19:54:30,164 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/7a0dfa548faa42c19d07fe7883244b92, entries=1, sequenceid=25, filesize=4.9 K 2023-03-08 19:54:30,165 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2d54b0ef5ff21a425fa7a6fc1850c764 in 828ms, sequenceid=25, compaction requested=true 2023-03-08 19:54:30,165 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2d54b0ef5ff21a425fa7a6fc1850c764: 2023-03-08 19:54:30,165 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. 2023-03-08 19:54:30,165 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:30,165 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:30,165 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:30,165 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 19:54:30,165 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-08 19:54:30,167 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:30,167 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:30,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:30,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:30,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:30,167 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:30,167 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:30,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:30,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:30,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:30,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:30,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:30,168 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 19:54:30,168 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@61d77907[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:30,168 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 19:54:30,169 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:30,170 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:30,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:30,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:30,170 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 19:54:30,170 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-08 19:54:30,170 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:30,170 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:30,170 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:30,172 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 19:54:30,172 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:30,172 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:30,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:30,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:30,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:30,172 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-08 19:54:30,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:30,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:30,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:30,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:30,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:30,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:30,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:30,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:30,174 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 19:54:30,174 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-08 19:54:30,174 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:30,174 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:30,174 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:30,175 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:30,176 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:30,176 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:30,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:30,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:30,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:30,176 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:30,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:30,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:30,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:30,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:30,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:30,177 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:30,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:30,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:30,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:30,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:30,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:30,178 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:30,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:30,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:30,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:30,180 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:30,180 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:30,180 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:30,180 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 19:54:30,180 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:30,180 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:30,180 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:30,180 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:30,180 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:30,180 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:30,180 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 19:54:30,180 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:30,180 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:30,180 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:30,180 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:30,180 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:30,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:30,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:30,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:30,280 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:30,281 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:30,285 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:30,287 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:30,287 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:30,287 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 19:54:30,287 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:30,287 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 19:54:30,287 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 19:54:30,288 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:30,288 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:30,289 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:30,289 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:30,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:30,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:30,289 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:30,289 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:30,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:30,290 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:30,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:30,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:30,290 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:30,290 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 19:54:30,290 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:30,290 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 19:54:30,290 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:30,290 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 19:54:30,291 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:30,291 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:30,291 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. 2023-03-08 19:54:30,291 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. started... 2023-03-08 19:54:30,291 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2d54b0ef5ff21a425fa7a6fc1850c764 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:54:30,706 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101896c28c89a124dbbaa74de25294f39b1 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101896c28c89a124dbbaa74de25294f39b1 2023-03-08 19:54:31,112 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/.tmp/family1/5afcffc764004006a7817fd2b24ba4dd 2023-03-08 19:54:31,117 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/.tmp/family1/5afcffc764004006a7817fd2b24ba4dd as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/5afcffc764004006a7817fd2b24ba4dd 2023-03-08 19:54:31,121 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/5afcffc764004006a7817fd2b24ba4dd, entries=1, sequenceid=29, filesize=4.9 K 2023-03-08 19:54:31,122 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2d54b0ef5ff21a425fa7a6fc1850c764 in 831ms, sequenceid=29, compaction requested=true 2023-03-08 19:54:31,122 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2d54b0ef5ff21a425fa7a6fc1850c764: 2023-03-08 19:54:31,122 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. 2023-03-08 19:54:31,122 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:31,122 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:31,122 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:31,122 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 19:54:31,122 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-08 19:54:31,124 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,124 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:31,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:31,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:31,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:31,124 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,125 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:31,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:31,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:31,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:31,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:31,126 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 19:54:31,126 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4b651c9c[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:31,126 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 19:54:31,126 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,127 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,127 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 19:54:31,127 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-08 19:54:31,127 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:31,127 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:31,127 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:31,129 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:31,129 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 19:54:31,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:31,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:31,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:31,129 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:31,129 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-08 19:54:31,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:31,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:31,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:31,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:31,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:31,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:31,131 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 19:54:31,131 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-08 19:54:31,131 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:31,131 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:31,131 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,132 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:31,133 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,133 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,133 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:31,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:31,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:31,133 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,134 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:31,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:31,134 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:31,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:31,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:31,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,135 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:31,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:31,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:31,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:31,139 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:31,139 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:31,140 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:31,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:31,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:31,140 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-08 19:54:31,140 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 19:54:31,140 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:31,140 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:31,139 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:31,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:31,140 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,140 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:31,140 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:31,140 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:31,140 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,140 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,140 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,240 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:31,241 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:31,246 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:31,248 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:31,248 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,248 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 19:54:31,248 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:31,248 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 19:54:31,248 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 19:54:31,248 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,248 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,252 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:31,252 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:31,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:31,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:31,253 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:31,253 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:31,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,253 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:31,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,254 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,254 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 19:54:31,254 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,254 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 19:54:31,254 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:31,254 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 19:54:31,254 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:31,254 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:31,254 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. 2023-03-08 19:54:31,254 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. started... 2023-03-08 19:54:31,254 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2d54b0ef5ff21a425fa7a6fc1850c764 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:54:31,266 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010334df226ca38441ba8cbcbb086d763af5 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2016010334df226ca38441ba8cbcbb086d763af5 2023-03-08 19:54:31,672 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/.tmp/family1/8086dc22e45e49098018148ec8133abc 2023-03-08 19:54:31,677 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/.tmp/family1/8086dc22e45e49098018148ec8133abc as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/8086dc22e45e49098018148ec8133abc 2023-03-08 19:54:31,681 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/8086dc22e45e49098018148ec8133abc, entries=1, sequenceid=33, filesize=4.9 K 2023-03-08 19:54:31,682 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2d54b0ef5ff21a425fa7a6fc1850c764 in 428ms, sequenceid=33, compaction requested=true 2023-03-08 19:54:31,682 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2d54b0ef5ff21a425fa7a6fc1850c764: 2023-03-08 19:54:31,682 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. 2023-03-08 19:54:31,682 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:31,682 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:31,682 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:31,682 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 19:54:31,682 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-08 19:54:31,684 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:31,684 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:31,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:31,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:31,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:31,684 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,684 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:31,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:31,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:31,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:31,685 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 19:54:31,685 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@be32d80[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:31,685 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 19:54:31,685 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,686 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,687 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 19:54:31,687 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-08 19:54:31,687 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:31,687 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:31,687 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:31,688 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:31,688 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 19:54:31,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:31,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:31,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:31,688 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:31,688 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-08 19:54:31,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:31,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:31,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:31,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:31,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:31,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:31,690 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 19:54:31,690 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-08 19:54:31,690 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:31,690 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:31,690 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,690 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:31,693 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,693 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:31,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,693 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:31,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:31,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:31,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:31,693 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:31,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:31,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:31,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,694 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:31,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:31,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:31,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:31,696 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:31,696 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,696 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,696 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:31,696 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,696 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,696 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,697 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:31,697 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 19:54:31,697 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:31,697 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:31,697 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:31,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:31,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:31,697 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:31,697 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:31,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:31,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:31,797 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:31,799 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:31,803 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:31,804 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:31,804 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,804 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 19:54:31,804 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:31,804 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 19:54:31,804 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 19:54:31,805 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,805 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,806 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:31,806 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:31,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:31,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:31,807 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:31,807 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:31,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,807 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:31,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,807 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,808 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 19:54:31,808 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:31,808 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 19:54:31,808 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:31,808 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 19:54:31,808 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:31,808 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:31,808 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. 2023-03-08 19:54:31,808 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. started... 2023-03-08 19:54:31,808 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2d54b0ef5ff21a425fa7a6fc1850c764 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:54:32,221 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090733efd2ce4e0b43fba2969f153f2f2f91 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015090733efd2ce4e0b43fba2969f153f2f2f91 2023-03-08 19:54:32,627 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/.tmp/family1/4061fba91dac423497adf5063fc499c6 2023-03-08 19:54:32,632 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/.tmp/family1/4061fba91dac423497adf5063fc499c6 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/4061fba91dac423497adf5063fc499c6 2023-03-08 19:54:32,636 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/4061fba91dac423497adf5063fc499c6, entries=1, sequenceid=37, filesize=4.9 K 2023-03-08 19:54:32,637 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2d54b0ef5ff21a425fa7a6fc1850c764 in 829ms, sequenceid=37, compaction requested=true 2023-03-08 19:54:32,637 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2d54b0ef5ff21a425fa7a6fc1850c764: 2023-03-08 19:54:32,637 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. 2023-03-08 19:54:32,637 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:32,637 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:32,637 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:32,637 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 19:54:32,637 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-08 19:54:32,639 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:32,639 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:32,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:32,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:32,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:32,639 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:32,639 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:32,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:32,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:32,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:32,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:32,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:32,641 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 19:54:32,641 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@426ae200[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:32,641 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 19:54:32,641 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:32,643 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:32,643 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:32,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:32,643 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 19:54:32,643 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-08 19:54:32,643 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:32,643 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:32,643 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:32,644 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 19:54:32,644 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:32,644 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:32,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:32,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:32,644 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-08 19:54:32,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:32,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:32,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:32,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:32,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:32,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:32,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:32,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:32,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:32,647 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 19:54:32,647 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-08 19:54:32,647 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:32,647 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:32,647 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:32,647 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:32,648 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:32,648 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:32,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:32,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:32,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:32,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:32,648 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:32,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:32,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:32,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:32,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:32,649 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:32,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:32,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:32,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:32,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:32,650 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:32,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:32,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:32,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:32,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:32,652 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:32,652 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:32,652 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:32,652 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:32,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:32,652 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 19:54:32,652 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:32,652 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:32,652 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:32,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:32,652 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:32,652 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:32,652 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:32,653 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:32,652 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:32,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:32,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:32,653 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:32,753 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:32,754 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:32,758 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:32,760 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:32,760 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:32,760 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 19:54:32,760 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:32,760 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-08 19:54:32,760 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-08 19:54:32,761 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:32,761 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:32,763 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:32,763 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:32,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:32,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:32,763 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:32,763 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:32,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:32,764 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:32,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:32,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:32,764 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:32,764 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-08 19:54:32,764 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:32,764 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-08 19:54:32,764 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:32,764 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-08 19:54:32,765 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:32,765 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:32,765 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. 2023-03-08 19:54:32,765 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. started... 2023-03-08 19:54:32,765 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2d54b0ef5ff21a425fa7a6fc1850c764 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:54:33,178 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511202a937432307d4d2b9c1d189cb490f3fc to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201511202a937432307d4d2b9c1d189cb490f3fc 2023-03-08 19:54:33,583 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/.tmp/family1/85683f1fa71b410c8c6354a44cba7847 2023-03-08 19:54:33,588 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/.tmp/family1/85683f1fa71b410c8c6354a44cba7847 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/85683f1fa71b410c8c6354a44cba7847 2023-03-08 19:54:33,592 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/85683f1fa71b410c8c6354a44cba7847, entries=1, sequenceid=41, filesize=4.9 K 2023-03-08 19:54:33,592 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2d54b0ef5ff21a425fa7a6fc1850c764 in 827ms, sequenceid=41, compaction requested=true 2023-03-08 19:54:33,593 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2d54b0ef5ff21a425fa7a6fc1850c764: 2023-03-08 19:54:33,593 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. 2023-03-08 19:54:33,593 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:33,593 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:33,593 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:33,593 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-08 19:54:33,593 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-08 19:54:33,595 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:33,595 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:33,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:33,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:33,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:33,595 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:33,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:33,595 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:33,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:33,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:33,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:33,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:33,596 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-08 19:54:33,596 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@182fb40b[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:33,596 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-08 19:54:33,596 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:33,598 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:33,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:33,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:33,598 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-08 19:54:33,598 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-08 19:54:33,598 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:33,598 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:33,598 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:33,599 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-08 19:54:33,599 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:33,599 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:33,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:33,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:33,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:33,599 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-08 19:54:33,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:33,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:33,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:33,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:33,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:33,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:33,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:33,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:33,601 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-08 19:54:33,601 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-08 19:54:33,601 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:33,602 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:33,602 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:33,602 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:33,605 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:33,605 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:33,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:33,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:33,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:33,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:33,605 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:33,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:33,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:33,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:33,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:33,605 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:33,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:33,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:33,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:33,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:33,606 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:33,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:33,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:33,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:33,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:33,614 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:33,614 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:33,614 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:33,614 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:33,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:33,614 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-08 19:54:33,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:33,614 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:33,614 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:33,614 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:33,615 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:33,615 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:33,615 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:33,615 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:33,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:33,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:33,615 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:33,615 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:33,715 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:33,716 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:33,717 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39989] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyPolicy 2023-03-08 19:54:33,719 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] procedure2.ProcedureExecutor(1029): Stored pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 19:54:33,720 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 19:54:33,721 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 19:54:33,721 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=38, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678305873721 2023-03-08 19:54:33,724 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 19:54:33,725 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyPolicy' and column 'family1' 2023-03-08 19:54:33,725 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 19:54:33,725 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] zookeeper.ReadOnlyZKClient(139): Connect 0x646f78e0 to 127.0.0.1:56751 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 19:54:33,737 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3398bebf, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 19:54:33,738 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 19:54:34,579 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103800562553e844f5fa7e7398beeaec68f to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103800562553e844f5fa7e7398beeaec68f 2023-03-08 19:54:34,580 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 19:54:34,592 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 19:54:34,594 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37160, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 19:54:34,600 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 19:54:34,601 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33656, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 19:54:34,610 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/fb7f9d7e61c14a7cb28968432f268037 first=Optional[r3] last=Optional[r6] 2023-03-08 19:54:34,611 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764., hostname=jenkins-hbase4.apache.org,33367,1678305218680, seqNum=2 for row with hfile group [{family1,hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/fb7f9d7e61c14a7cb28968432f268037}] 2023-03-08 19:54:34,616 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(763): Validating hfile at hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/fb7f9d7e61c14a7cb28968432f268037 for inclusion in 2d54b0ef5ff21a425fa7a6fc1850c764/family1 2023-03-08 19:54:34,619 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-08 19:54:34,619 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 19:54:34,619 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(2446): Flush status journal for 2d54b0ef5ff21a425fa7a6fc1850c764: 2023-03-08 19:54:34,620 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/fb7f9d7e61c14a7cb28968432f268037 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyPolicy__6rb9c6kd7a9tse5f650p1elkvu1ebjjv1ue1t6mlk49aq4khtlb73oa33l7ku3co/family1/fb7f9d7e61c14a7cb28968432f268037 2023-03-08 19:54:34,622 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyPolicy__6rb9c6kd7a9tse5f650p1elkvu1ebjjv1ue1t6mlk49aq4khtlb73oa33l7ku3co/family1/fb7f9d7e61c14a7cb28968432f268037 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/5467f99f52224f7bad3a6776a66cc21c_SeqId_44_ 2023-03-08 19:54:34,623 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(849): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyPolicy__6rb9c6kd7a9tse5f650p1elkvu1ebjjv1ue1t6mlk49aq4khtlb73oa33l7ku3co/family1/fb7f9d7e61c14a7cb28968432f268037 into 2d54b0ef5ff21a425fa7a6fc1850c764/family1 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/5467f99f52224f7bad3a6776a66cc21c_SeqId_44_ - updating store file list. 2023-03-08 19:54:34,626 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(882): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/5467f99f52224f7bad3a6776a66cc21c_SeqId_44_ into 2d54b0ef5ff21a425fa7a6fc1850c764/family1 2023-03-08 19:54:34,626 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(855): Successfully loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyPolicy__6rb9c6kd7a9tse5f650p1elkvu1ebjjv1ue1t6mlk49aq4khtlb73oa33l7ku3co/family1/fb7f9d7e61c14a7cb28968432f268037 into 2d54b0ef5ff21a425fa7a6fc1850c764/family1 (new location: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/5467f99f52224f7bad3a6776a66cc21c_SeqId_44_) 2023-03-08 19:54:34,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyPolicy__6rb9c6kd7a9tse5f650p1elkvu1ebjjv1ue1t6mlk49aq4khtlb73oa33l7ku3co/family1/fb7f9d7e61c14a7cb28968432f268037 2023-03-08 19:54:34,627 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-08 19:54:34,649 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 19:54:34,649 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1b538d9e to 127.0.0.1:56751 2023-03-08 19:54:34,649 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 19:54:34,649 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 19:54:34,649 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x64c3cb99 to 127.0.0.1:56751 2023-03-08 19:54:34,650 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 19:54:34,650 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 19:54:34,650 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6dec2b43 to 127.0.0.1:56751 2023-03-08 19:54:34,650 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 19:54:34,653 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-08 19:54:34,653 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(1912): 2d54b0ef5ff21a425fa7a6fc1850c764/family1 is initiating minor compaction 2023-03-08 19:54:34,653 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@1ebb9006; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 19:54:34,653 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 2d54b0ef5ff21a425fa7a6fc1850c764/family1 in testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. 2023-03-08 19:54:34,653 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/aedba52f549e45ae85769a5c7a88b987, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/9fe74ddadac843ccb5845ca688f25c9c, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/de38a040076048ceb34ac6d5eabe452e, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/3c979f53dcbd40eaa6ecdaa6ec42421b, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/7a0dfa548faa42c19d07fe7883244b92, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/5afcffc764004006a7817fd2b24ba4dd, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/8086dc22e45e49098018148ec8133abc, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/4061fba91dac423497adf5063fc499c6, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/85683f1fa71b410c8c6354a44cba7847, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/5467f99f52224f7bad3a6776a66cc21c_SeqId_44_] into tmpdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/.tmp, totalSize=50.0 K 2023-03-08 19:54:34,654 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting aedba52f549e45ae85769a5c7a88b987, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-08 19:54:34,654 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 9fe74ddadac843ccb5845ca688f25c9c, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-08 19:54:34,654 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting de38a040076048ceb34ac6d5eabe452e, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-08 19:54:34,654 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 3c979f53dcbd40eaa6ecdaa6ec42421b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-08 19:54:34,654 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 7a0dfa548faa42c19d07fe7883244b92, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-08 19:54:34,654 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 5afcffc764004006a7817fd2b24ba4dd, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-08 19:54:34,654 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 8086dc22e45e49098018148ec8133abc, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-08 19:54:34,654 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 4061fba91dac423497adf5063fc499c6, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-08 19:54:34,654 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 85683f1fa71b410c8c6354a44cba7847, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-08 19:54:34,654 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 5467f99f52224f7bad3a6776a66cc21c_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-08 19:54:34,655 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 19:54:34,656 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:54:34,658 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151228dc5a9cedef8549f8ab3d2081a8ac79f2 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151228dc5a9cedef8549f8ab3d2081a8ac79f2 2023-03-08 19:54:34,659 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231afe314b5fc974827b1725a58a891e5bd to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231afe314b5fc974827b1725a58a891e5bd 2023-03-08 19:54:34,661 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101896c28c89a124dbbaa74de25294f39b1 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101896c28c89a124dbbaa74de25294f39b1 2023-03-08 19:54:34,662 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2016010334df226ca38441ba8cbcbb086d763af5 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2016010334df226ca38441ba8cbcbb086d763af5 2023-03-08 19:54:34,662 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-03-08 19:54:34,662 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-08 19:54:34,662 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 19:54:34,662 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 19:54:34,662 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 19:54:34,662 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 19:54:34,663 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 19:54:34,663 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 19:54:34,663 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 19:54:34,663 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 19:54:34,711 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] throttle.PressureAwareThroughputController(145): 2d54b0ef5ff21a425fa7a6fc1850c764#family1#compaction#49 average throughput is 0.45 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 19:54:35,530 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/.tmp/family1/d0937e6818de42ef808906225e085f82 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/d0937e6818de42ef808906225e085f82 2023-03-08 19:54:35,535 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 2d54b0ef5ff21a425fa7a6fc1850c764/family1 of 2d54b0ef5ff21a425fa7a6fc1850c764 into d0937e6818de42ef808906225e085f82(size=6.6 K), total size for store is 171.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 19:54:35,535 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 2d54b0ef5ff21a425fa7a6fc1850c764: 2023-03-08 19:54:35,535 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1678305274653; duration=0sec 2023-03-08 19:54:35,536 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 19:54:35,542 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205b890d2c963694336892f129b864447a2 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205b890d2c963694336892f129b864447a2 2023-03-08 19:54:35,543 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 19:54:35,567 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7c3ae825a1b34e4cab3480d0a01984c5 first=Optional[r00] last=Optional[r2] 2023-03-08 19:54:35,568 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764., hostname=jenkins-hbase4.apache.org,33367,1678305218680, seqNum=2 for row with hfile group [{family1,hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7c3ae825a1b34e4cab3480d0a01984c5}] 2023-03-08 19:54:35,570 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(763): Validating hfile at hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7c3ae825a1b34e4cab3480d0a01984c5 for inclusion in 2d54b0ef5ff21a425fa7a6fc1850c764/family1 2023-03-08 19:54:35,572 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-08 19:54:35,572 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 19:54:35,572 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(2446): Flush status journal for 2d54b0ef5ff21a425fa7a6fc1850c764: 2023-03-08 19:54:35,573 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7c3ae825a1b34e4cab3480d0a01984c5 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyPolicy__ifha8pijnfolos6bred4tohio9vtgbjan0lkpb883evqbb5nfvos64l8p72q4g3d/family1/7c3ae825a1b34e4cab3480d0a01984c5 2023-03-08 19:54:35,575 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyPolicy__ifha8pijnfolos6bred4tohio9vtgbjan0lkpb883evqbb5nfvos64l8p72q4g3d/family1/7c3ae825a1b34e4cab3480d0a01984c5 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/3b97a328b90249d5b8694917f18ce560_SeqId_47_ 2023-03-08 19:54:35,576 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(849): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyPolicy__ifha8pijnfolos6bred4tohio9vtgbjan0lkpb883evqbb5nfvos64l8p72q4g3d/family1/7c3ae825a1b34e4cab3480d0a01984c5 into 2d54b0ef5ff21a425fa7a6fc1850c764/family1 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/3b97a328b90249d5b8694917f18ce560_SeqId_47_ - updating store file list. 2023-03-08 19:54:35,580 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(882): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/3b97a328b90249d5b8694917f18ce560_SeqId_47_ into 2d54b0ef5ff21a425fa7a6fc1850c764/family1 2023-03-08 19:54:35,580 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(855): Successfully loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyPolicy__ifha8pijnfolos6bred4tohio9vtgbjan0lkpb883evqbb5nfvos64l8p72q4g3d/family1/7c3ae825a1b34e4cab3480d0a01984c5 into 2d54b0ef5ff21a425fa7a6fc1850c764/family1 (new location: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/3b97a328b90249d5b8694917f18ce560_SeqId_47_) 2023-03-08 19:54:35,580 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyPolicy__ifha8pijnfolos6bred4tohio9vtgbjan0lkpb883evqbb5nfvos64l8p72q4g3d/family1/7c3ae825a1b34e4cab3480d0a01984c5 2023-03-08 19:54:35,580 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-08 19:54:35,581 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 345120 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-08 19:54:35,581 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(1912): 2d54b0ef5ff21a425fa7a6fc1850c764/family1 is initiating minor compaction (all files) 2023-03-08 19:54:35,581 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@683c1bc3; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 19:54:35,581 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 2d54b0ef5ff21a425fa7a6fc1850c764/family1 in testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. 2023-03-08 19:54:35,582 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/a9ac14e4aba34b548aad689d0dc09957, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/d0937e6818de42ef808906225e085f82, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/3b97a328b90249d5b8694917f18ce560_SeqId_47_] into tmpdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/.tmp, totalSize=337.0 K 2023-03-08 19:54:35,582 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting a9ac14e4aba34b548aad689d0dc09957, keycount=1000, bloomtype=ROW, size=165.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878843488 2023-03-08 19:54:35,583 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting d0937e6818de42ef808906225e085f82, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621243488 2023-03-08 19:54:35,583 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 3b97a328b90249d5b8694917f18ce560_SeqId_47_, keycount=1002, bloomtype=ROW, size=165.4 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878843488 2023-03-08 19:54:35,584 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 19:54:35,584 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:54:35,586 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015113009bf59d4ebd44cdb96d6d218cea3fd74 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015113009bf59d4ebd44cdb96d6d218cea3fd74 2023-03-08 19:54:35,587 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512010c6a785f419c4d1ca49c97abafb14d48 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512010c6a785f419c4d1ca49c97abafb14d48 2023-03-08 19:54:35,589 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512059480d3f4139d452aa7c93f5821e834f6 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512059480d3f4139d452aa7c93f5821e834f6 2023-03-08 19:54:35,589 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 3 expired mob files are deleted 2023-03-08 19:54:35,589 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-08 19:54:35,589 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-08 19:54:35,589 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-08 19:54:35,589 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 19:54:35,591 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=38, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 19:54:35,592 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=38, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.8730 sec 2023-03-08 19:54:35,603 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] throttle.PressureAwareThroughputController(145): 2d54b0ef5ff21a425fa7a6fc1850c764#family1#compaction#50 average throughput is 14.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 19:54:35,628 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/aedba52f549e45ae85769a5c7a88b987, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/9fe74ddadac843ccb5845ca688f25c9c, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/de38a040076048ceb34ac6d5eabe452e, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/3c979f53dcbd40eaa6ecdaa6ec42421b, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/7a0dfa548faa42c19d07fe7883244b92, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/5afcffc764004006a7817fd2b24ba4dd, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/8086dc22e45e49098018148ec8133abc, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/4061fba91dac423497adf5063fc499c6, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/85683f1fa71b410c8c6354a44cba7847, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/5467f99f52224f7bad3a6776a66cc21c_SeqId_44_] to archive 2023-03-08 19:54:35,629 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:54:35,631 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/aedba52f549e45ae85769a5c7a88b987 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/aedba52f549e45ae85769a5c7a88b987 2023-03-08 19:54:35,633 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/9fe74ddadac843ccb5845ca688f25c9c to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/9fe74ddadac843ccb5845ca688f25c9c 2023-03-08 19:54:35,636 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/de38a040076048ceb34ac6d5eabe452e to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/de38a040076048ceb34ac6d5eabe452e 2023-03-08 19:54:35,639 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/3c979f53dcbd40eaa6ecdaa6ec42421b to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/3c979f53dcbd40eaa6ecdaa6ec42421b 2023-03-08 19:54:35,641 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/7a0dfa548faa42c19d07fe7883244b92 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/7a0dfa548faa42c19d07fe7883244b92 2023-03-08 19:54:35,645 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/5afcffc764004006a7817fd2b24ba4dd to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/5afcffc764004006a7817fd2b24ba4dd 2023-03-08 19:54:35,647 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/8086dc22e45e49098018148ec8133abc to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/8086dc22e45e49098018148ec8133abc 2023-03-08 19:54:35,648 DEBUG [hfile_cleaner-dir-scan-pool-0] cleaner.CleanerChore(483): Couldn't delete 'hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyPolicy' yet because it isn't empty w/exception. org.apache.hadoop.fs.PathIsNotEmptyDirectoryException: ``/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyPolicy is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) ': Directory is not empty: `/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyPolicy is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:121) at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:88) at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:1595) at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:881) at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:878) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.delete(DistributedFileSystem.java:878) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$null$4(CleanerChore.java:453) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.deleteAction(CleanerChore.java:478) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:453) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:455) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:455) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:792) at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2153) at org.apache.hadoop.hbase.util.FutureUtils.addListener(FutureUtils.java:61) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:439) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$null$2(CleanerChore.java:432) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.PathIsNotEmptyDirectoryException): `/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyPolicy is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1540) at org.apache.hadoop.ipc.Client.call(Client.java:1486) at org.apache.hadoop.ipc.Client.call(Client.java:1385) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118) at com.sun.proxy.$Proxy29.delete(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete(ClientNamenodeProtocolTranslatorPB.java:572) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359) at com.sun.proxy.$Proxy30.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:1593) ... 29 more 2023-03-08 19:54:35,650 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/4061fba91dac423497adf5063fc499c6 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/4061fba91dac423497adf5063fc499c6 2023-03-08 19:54:35,651 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/85683f1fa71b410c8c6354a44cba7847 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/85683f1fa71b410c8c6354a44cba7847 2023-03-08 19:54:35,653 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/5467f99f52224f7bad3a6776a66cc21c_SeqId_44_ to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/5467f99f52224f7bad3a6776a66cc21c_SeqId_44_ 2023-03-08 19:54:35,706 INFO [Listener at localhost/34049] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=256 (was 255) - Thread LEAK? -, OpenFileDescriptor=610 (was 610), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=204 (was 233), ProcessCount=170 (was 170), AvailableMemoryMB=8351 (was 8280) - AvailableMemoryMB LEAK? - 2023-03-08 19:54:35,721 INFO [Listener at localhost/34049] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=256, OpenFileDescriptor=610, MaxFileDescriptor=60000, SystemLoadAverage=204, ProcessCount=170, AvailableMemoryMB=8350 2023-03-08 19:54:35,724 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-08 19:54:35,725 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure2.ProcedureExecutor(1029): Stored pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:35,726 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 19:54:35,726 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly" procId is: 39 2023-03-08 19:54:35,727 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 19:54:35,727 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-08 19:54:35,728 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d 2023-03-08 19:54:35,729 DEBUG [HFileArchiver-9] backup.HFileArchiver(153): Directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d empty. 2023-03-08 19:54:35,729 DEBUG [HFileArchiver-9] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d 2023-03-08 19:54:35,729 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly regions 2023-03-08 19:54:35,828 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-08 19:54:36,029 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-08 19:54:36,138 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-03-08 19:54:36,139 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(7675): creating {ENCODED => c1edfa28825217aa75bd370b97cfc23d, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp 2023-03-08 19:54:36,330 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-08 19:54:36,552 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:54:36,552 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1604): Closing c1edfa28825217aa75bd370b97cfc23d, disabling compactions & flushes 2023-03-08 19:54:36,552 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:36,552 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:36,552 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. after waiting 0 ms 2023-03-08 19:54:36,552 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:36,552 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:36,552 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1558): Region close journal for c1edfa28825217aa75bd370b97cfc23d: 2023-03-08 19:54:36,555 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 19:54:36,556 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678305276555"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678305276555"}]},"ts":"1678305276555"} 2023-03-08 19:54:36,557 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-08 19:54:36,558 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 19:54:36,558 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678305276558"}]},"ts":"1678305276558"} 2023-03-08 19:54:36,559 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLING in hbase:meta 2023-03-08 19:54:36,562 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=c1edfa28825217aa75bd370b97cfc23d, ASSIGN}] 2023-03-08 19:54:36,563 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=c1edfa28825217aa75bd370b97cfc23d, ASSIGN 2023-03-08 19:54:36,564 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=c1edfa28825217aa75bd370b97cfc23d, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33367,1678305218680; forceNewPlan=false, retain=false 2023-03-08 19:54:36,715 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=c1edfa28825217aa75bd370b97cfc23d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:36,715 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678305276715"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678305276715"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678305276715"}]},"ts":"1678305276715"} 2023-03-08 19:54:36,717 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=41, ppid=40, state=RUNNABLE; OpenRegionProcedure c1edfa28825217aa75bd370b97cfc23d, server=jenkins-hbase4.apache.org,33367,1678305218680}] 2023-03-08 19:54:36,831 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-08 19:54:36,835 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/.tmp/family1/66b0346ceec34e598f40f1c45ff9ff9e as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/66b0346ceec34e598f40f1c45ff9ff9e 2023-03-08 19:54:36,840 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 2d54b0ef5ff21a425fa7a6fc1850c764/family1 of 2d54b0ef5ff21a425fa7a6fc1850c764 into 66b0346ceec34e598f40f1c45ff9ff9e(size=166.4 K), total size for store is 166.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 19:54:36,840 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 2d54b0ef5ff21a425fa7a6fc1850c764: 2023-03-08 19:54:36,840 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764., storeName=family1, fileCount=3, fileSize=337.0 K (165.1 K, 6.6 K, 165.4 K), priority=2, time=1678305275581; duration=1sec 2023-03-08 19:54:36,840 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 19:54:36,871 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:36,871 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c1edfa28825217aa75bd370b97cfc23d, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d.', STARTKEY => '', ENDKEY => ''} 2023-03-08 19:54:36,872 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly c1edfa28825217aa75bd370b97cfc23d 2023-03-08 19:54:36,872 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:54:36,872 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for c1edfa28825217aa75bd370b97cfc23d 2023-03-08 19:54:36,872 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for c1edfa28825217aa75bd370b97cfc23d 2023-03-08 19:54:36,873 INFO [StoreOpener-c1edfa28825217aa75bd370b97cfc23d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region c1edfa28825217aa75bd370b97cfc23d 2023-03-08 19:54:36,874 DEBUG [StoreOpener-c1edfa28825217aa75bd370b97cfc23d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1 2023-03-08 19:54:36,874 DEBUG [StoreOpener-c1edfa28825217aa75bd370b97cfc23d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1 2023-03-08 19:54:36,875 INFO [StoreOpener-c1edfa28825217aa75bd370b97cfc23d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c1edfa28825217aa75bd370b97cfc23d columnFamilyName family1 2023-03-08 19:54:36,876 INFO [StoreOpener-c1edfa28825217aa75bd370b97cfc23d-1] regionserver.HStore(310): Store=c1edfa28825217aa75bd370b97cfc23d/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:54:36,877 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d 2023-03-08 19:54:36,877 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d 2023-03-08 19:54:36,880 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for c1edfa28825217aa75bd370b97cfc23d 2023-03-08 19:54:36,881 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 19:54:36,882 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened c1edfa28825217aa75bd370b97cfc23d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66760139, jitterRate=-0.005196407437324524}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 19:54:36,882 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for c1edfa28825217aa75bd370b97cfc23d: 2023-03-08 19:54:36,883 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d., pid=41, masterSystemTime=1678305276868 2023-03-08 19:54:36,884 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:36,884 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:36,885 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=c1edfa28825217aa75bd370b97cfc23d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:36,885 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678305276884"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678305276884"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678305276884"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678305276884"}]},"ts":"1678305276884"} 2023-03-08 19:54:36,887 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=41, resume processing ppid=40 2023-03-08 19:54:36,888 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=41, ppid=40, state=SUCCESS; OpenRegionProcedure c1edfa28825217aa75bd370b97cfc23d, server=jenkins-hbase4.apache.org,33367,1678305218680 in 169 msec 2023-03-08 19:54:36,889 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=40, resume processing ppid=39 2023-03-08 19:54:36,889 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=40, ppid=39, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=c1edfa28825217aa75bd370b97cfc23d, ASSIGN in 326 msec 2023-03-08 19:54:36,890 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 19:54:36,890 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678305276890"}]},"ts":"1678305276890"} 2023-03-08 19:54:36,891 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLED in hbase:meta 2023-03-08 19:54:36,893 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 19:54:36,894 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.1690 sec 2023-03-08 19:54:36,911 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/a9ac14e4aba34b548aad689d0dc09957, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/d0937e6818de42ef808906225e085f82, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/3b97a328b90249d5b8694917f18ce560_SeqId_47_] to archive 2023-03-08 19:54:36,912 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:54:36,914 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/a9ac14e4aba34b548aad689d0dc09957 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/a9ac14e4aba34b548aad689d0dc09957 2023-03-08 19:54:36,915 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/d0937e6818de42ef808906225e085f82 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/d0937e6818de42ef808906225e085f82 2023-03-08 19:54:36,917 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/3b97a328b90249d5b8694917f18ce560_SeqId_47_ to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/family1/3b97a328b90249d5b8694917f18ce560_SeqId_47_ 2023-03-08 19:54:37,832 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-08 19:54:37,833 INFO [Listener at localhost/34049] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, procId: 39 completed 2023-03-08 19:54:37,856 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:37,858 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:37,858 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 19:54:37,858 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:37,858 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-08 19:54:37,858 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-08 19:54:37,858 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:37,859 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:37,860 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:37,860 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:37,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:37,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:37,860 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:37,860 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:37,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:37,861 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:37,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:37,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:37,861 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:37,861 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:37,862 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-08 19:54:37,862 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:37,862 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-08 19:54:37,862 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:37,862 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:37,862 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:37,862 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. started... 2023-03-08 19:54:37,863 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c1edfa28825217aa75bd370b97cfc23d 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-08 19:54:38,289 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130c7821063676e45e4bf7bbc9a7558f089 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130c7821063676e45e4bf7bbc9a7558f089 2023-03-08 19:54:38,693 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/.tmp/family1/9aadb18f51df4e97a73e6367195c7aa3 2023-03-08 19:54:38,699 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/.tmp/family1/9aadb18f51df4e97a73e6367195c7aa3 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/9aadb18f51df4e97a73e6367195c7aa3 2023-03-08 19:54:38,703 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/9aadb18f51df4e97a73e6367195c7aa3, entries=1000, sequenceid=5, filesize=191.4 K 2023-03-08 19:54:38,704 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for c1edfa28825217aa75bd370b97cfc23d in 841ms, sequenceid=5, compaction requested=false 2023-03-08 19:54:38,704 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 19:54:38,704 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c1edfa28825217aa75bd370b97cfc23d: 2023-03-08 19:54:38,704 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:38,704 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:38,704 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:38,704 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:38,705 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-08 19:54:38,705 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-08 19:54:38,706 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:38,706 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:38,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:38,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:38,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:38,707 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:38,707 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:38,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:38,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:38,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:38,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:38,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:38,708 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-08 19:54:38,708 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7bcdca6e[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:38,708 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-08 19:54:38,708 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:38,711 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:38,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:38,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:38,711 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:38,711 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-08 19:54:38,711 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-08 19:54:38,711 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:38,711 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:38,713 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:38,713 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-08 19:54:38,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:38,713 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:38,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:38,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:38,713 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-08 19:54:38,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:38,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:38,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:38,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:38,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:38,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:38,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:38,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:38,715 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-08 19:54:38,715 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-08 19:54:38,715 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:38,715 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:38,715 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:38,715 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:38,716 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:38,716 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:38,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:38,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:38,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:38,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:38,717 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:38,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:38,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:38,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:38,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:38,717 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:38,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:38,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:38,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:38,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:38,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:38,718 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:38,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:38,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:38,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:38,720 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:38,720 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:38,720 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:38,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:38,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:38,721 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 19:54:38,721 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:38,720 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:38,721 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:38,721 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:38,721 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:38,721 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:38,721 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:38,721 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:38,721 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:38,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:38,721 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:38,721 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:38,821 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:38,822 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:38,826 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:38,828 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:38,828 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:38,828 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 19:54:38,828 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:38,828 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-08 19:54:38,828 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-08 19:54:38,829 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:38,829 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:38,831 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:38,831 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:38,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:38,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:38,831 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:38,831 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:38,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:38,832 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:38,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:38,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:38,832 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:38,832 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-08 19:54:38,832 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:38,832 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-08 19:54:38,832 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:38,832 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-08 19:54:38,833 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:38,833 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:38,833 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:38,833 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. started... 2023-03-08 19:54:38,833 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c1edfa28825217aa75bd370b97cfc23d 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-08 19:54:39,246 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128b3b1e64cd21b4c7ea508978d3290ffc6 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128b3b1e64cd21b4c7ea508978d3290ffc6 2023-03-08 19:54:39,251 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/.tmp/family1/b364ac349c97413e8208c15ad6755201 2023-03-08 19:54:39,256 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/.tmp/family1/b364ac349c97413e8208c15ad6755201 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/b364ac349c97413e8208c15ad6755201 2023-03-08 19:54:39,261 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/b364ac349c97413e8208c15ad6755201, entries=1, sequenceid=9, filesize=5.0 K 2023-03-08 19:54:39,261 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for c1edfa28825217aa75bd370b97cfc23d in 428ms, sequenceid=9, compaction requested=false 2023-03-08 19:54:39,262 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c1edfa28825217aa75bd370b97cfc23d: 2023-03-08 19:54:39,262 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:39,262 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:39,262 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:39,262 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:39,262 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-08 19:54:39,262 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-08 19:54:39,264 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:39,264 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:39,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:39,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:39,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:39,264 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:39,264 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:39,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:39,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:39,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:39,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:39,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:39,266 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-08 19:54:39,266 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@69d067ba[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:39,266 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-08 19:54:39,266 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:39,267 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:39,267 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:39,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:39,267 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-08 19:54:39,267 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:39,267 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-08 19:54:39,267 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:39,267 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:39,269 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-08 19:54:39,269 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:39,269 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:39,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:39,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:39,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:39,269 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-08 19:54:39,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:39,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:39,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:39,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:39,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:39,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:39,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:39,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:39,272 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-08 19:54:39,272 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-08 19:54:39,272 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:39,272 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:39,272 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:39,272 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:39,274 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:39,274 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:39,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:39,274 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:39,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:39,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:39,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:39,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:39,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:39,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:39,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:39,275 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:39,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:39,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:39,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:39,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:39,276 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:39,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:39,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:39,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:39,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:39,280 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:39,281 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:39,281 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:39,281 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:39,281 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 19:54:39,281 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:39,281 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:39,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:39,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:39,281 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:39,281 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-08 19:54:39,281 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:39,281 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:39,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:39,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:39,281 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:39,281 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:39,281 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:39,282 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:39,382 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:39,383 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:39,386 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:39,388 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:39,389 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:39,389 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 19:54:39,389 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:39,389 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-08 19:54:39,389 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-08 19:54:39,389 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:39,389 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:39,391 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:39,391 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:39,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:39,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:39,391 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:39,391 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:39,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:39,391 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:39,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:39,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:39,392 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:39,392 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-08 19:54:39,392 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:39,392 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-08 19:54:39,392 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:39,392 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-08 19:54:39,392 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:39,392 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:39,392 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:39,392 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. started... 2023-03-08 19:54:39,393 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c1edfa28825217aa75bd370b97cfc23d 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:54:39,805 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120179a651cb293043829ea2b480da16305d to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120179a651cb293043829ea2b480da16305d 2023-03-08 19:54:40,210 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/.tmp/family1/ccd761b0780f45158f9dd1b14525b38d 2023-03-08 19:54:40,215 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/.tmp/family1/ccd761b0780f45158f9dd1b14525b38d as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/ccd761b0780f45158f9dd1b14525b38d 2023-03-08 19:54:40,218 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/ccd761b0780f45158f9dd1b14525b38d, entries=1, sequenceid=13, filesize=5.0 K 2023-03-08 19:54:40,219 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c1edfa28825217aa75bd370b97cfc23d in 826ms, sequenceid=13, compaction requested=true 2023-03-08 19:54:40,219 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c1edfa28825217aa75bd370b97cfc23d: 2023-03-08 19:54:40,219 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:40,219 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:40,219 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:40,219 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:40,219 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-08 19:54:40,219 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-08 19:54:40,222 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:40,222 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:40,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:40,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:40,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:40,223 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:40,223 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:40,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:40,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:40,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:40,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:40,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:40,225 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-08 19:54:40,225 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7cbfe262[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:40,225 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-08 19:54:40,225 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:40,227 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:40,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:40,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:40,228 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-08 19:54:40,228 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-08 19:54:40,228 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:40,228 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:40,228 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:40,230 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:40,230 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-08 19:54:40,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:40,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:40,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:40,230 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:40,230 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-08 19:54:40,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:40,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:40,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:40,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:40,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:40,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:40,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:40,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:40,232 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-08 19:54:40,232 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-08 19:54:40,232 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:40,232 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:40,232 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:40,232 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:40,235 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:40,235 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:40,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:40,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:40,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:40,235 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:40,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:40,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:40,235 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:40,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:40,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:40,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:40,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:40,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:40,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:40,236 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:40,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:40,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:40,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:40,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:40,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:40,239 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:40,240 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:40,240 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:40,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:40,240 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 19:54:40,240 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:40,240 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:40,240 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:40,240 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:40,240 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-08 19:54:40,240 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:40,240 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:40,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:40,240 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:40,240 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:40,240 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:40,240 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:40,241 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:40,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:40,340 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:40,341 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:40,345 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:40,347 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:40,347 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:40,347 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 19:54:40,347 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:40,347 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-08 19:54:40,347 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-08 19:54:40,348 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:40,348 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:40,349 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:40,349 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:40,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:40,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:40,350 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:40,350 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:40,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:40,350 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:40,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:40,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:40,350 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:40,351 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-08 19:54:40,351 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:40,351 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-08 19:54:40,351 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:40,351 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-08 19:54:40,351 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:40,351 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:40,351 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:40,351 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. started... 2023-03-08 19:54:40,351 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c1edfa28825217aa75bd370b97cfc23d 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:54:40,765 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512054f41dbf198ed4eb9a948421f765d631b to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512054f41dbf198ed4eb9a948421f765d631b 2023-03-08 19:54:40,801 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-03-08 19:54:41,171 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/.tmp/family1/4041e25692384f6ab2e15ec6bb6a8e59 2023-03-08 19:54:41,176 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/.tmp/family1/4041e25692384f6ab2e15ec6bb6a8e59 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/4041e25692384f6ab2e15ec6bb6a8e59 2023-03-08 19:54:41,180 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/4041e25692384f6ab2e15ec6bb6a8e59, entries=1, sequenceid=17, filesize=5.0 K 2023-03-08 19:54:41,181 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c1edfa28825217aa75bd370b97cfc23d in 830ms, sequenceid=17, compaction requested=true 2023-03-08 19:54:41,181 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c1edfa28825217aa75bd370b97cfc23d: 2023-03-08 19:54:41,181 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:41,181 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:41,181 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:41,181 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:41,181 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-08 19:54:41,181 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-08 19:54:41,184 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:41,184 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:41,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:41,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:41,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:41,184 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:41,184 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:41,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:41,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:41,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:41,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:41,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:41,186 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-08 19:54:41,186 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@19559e12[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:41,186 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-08 19:54:41,186 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:41,189 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:41,189 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:41,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:41,189 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-08 19:54:41,189 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:41,189 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:41,189 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-08 19:54:41,189 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:41,193 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-08 19:54:41,193 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:41,193 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:41,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:41,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:41,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:41,193 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-08 19:54:41,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:41,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:41,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:41,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:41,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:41,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:41,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:41,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:41,195 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-08 19:54:41,195 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-08 19:54:41,195 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:41,195 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:41,195 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:41,195 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:41,199 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:41,199 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:41,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:41,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:41,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:41,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:41,199 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:41,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:41,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:41,199 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:41,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:41,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:41,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:41,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:41,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:41,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:41,200 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:41,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:41,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:41,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:41,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:41,205 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:41,205 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:41,205 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:41,205 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 19:54:41,205 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:41,205 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:41,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:41,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:41,205 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:41,205 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:41,205 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:41,205 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:41,205 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:41,205 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:41,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:41,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:41,205 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:41,205 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:41,306 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:41,307 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:41,311 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:41,313 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:41,313 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:41,313 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 19:54:41,313 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:41,313 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-08 19:54:41,313 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-08 19:54:41,313 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:41,313 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:41,318 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:41,318 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:41,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:41,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:41,319 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:41,319 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:41,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:41,319 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:41,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:41,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:41,319 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:41,320 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-08 19:54:41,320 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:41,320 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-08 19:54:41,320 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:41,320 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-08 19:54:41,320 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:41,320 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:41,320 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:41,320 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. started... 2023-03-08 19:54:41,320 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c1edfa28825217aa75bd370b97cfc23d 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:54:41,733 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228d00ee7dcf570452aa657098dcd7c9ce3 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151228d00ee7dcf570452aa657098dcd7c9ce3 2023-03-08 19:54:42,139 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/.tmp/family1/c87aee1ea61445c9b1270a9d4ce5ec7a 2023-03-08 19:54:42,144 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/.tmp/family1/c87aee1ea61445c9b1270a9d4ce5ec7a as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/c87aee1ea61445c9b1270a9d4ce5ec7a 2023-03-08 19:54:42,148 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/c87aee1ea61445c9b1270a9d4ce5ec7a, entries=1, sequenceid=21, filesize=5.0 K 2023-03-08 19:54:42,149 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c1edfa28825217aa75bd370b97cfc23d in 829ms, sequenceid=21, compaction requested=true 2023-03-08 19:54:42,149 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c1edfa28825217aa75bd370b97cfc23d: 2023-03-08 19:54:42,149 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:42,149 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:42,149 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:42,149 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:42,149 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-08 19:54:42,149 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-08 19:54:42,153 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:42,153 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:42,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:42,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:42,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:42,154 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:42,154 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:42,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:42,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:42,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:42,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:42,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:42,155 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-08 19:54:42,155 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@cdea9ec[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:42,155 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-08 19:54:42,155 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:42,156 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:42,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:42,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:42,157 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-08 19:54:42,157 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-08 19:54:42,157 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:42,157 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:42,157 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:42,159 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-08 19:54:42,159 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:42,159 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:42,159 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-08 19:54:42,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:42,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:42,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:42,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:42,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:42,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:42,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:42,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:42,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:42,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:42,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:42,161 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-08 19:54:42,161 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-08 19:54:42,161 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:42,161 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:42,161 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:42,161 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:42,163 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:42,163 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:42,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:42,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:42,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:42,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:42,163 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:42,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:42,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:42,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:42,163 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:42,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:42,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:42,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:42,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:42,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:42,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:42,164 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:42,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:42,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:42,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:42,167 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:42,167 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:42,167 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:42,167 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:42,167 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:42,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:42,167 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 19:54:42,167 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:42,167 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:42,167 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:42,167 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:42,167 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:42,167 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-08 19:54:42,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:42,167 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:42,167 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:42,167 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:42,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:42,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:42,268 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:42,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:42,272 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:42,274 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:42,274 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:42,274 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 19:54:42,274 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:42,274 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-08 19:54:42,274 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-08 19:54:42,274 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:42,275 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:42,276 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:42,276 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:42,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:42,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:42,276 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:42,276 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:42,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:42,277 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:42,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:42,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:42,277 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:42,277 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-08 19:54:42,277 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:42,277 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-08 19:54:42,277 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:42,277 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-08 19:54:42,277 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:42,277 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:42,278 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:42,278 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. started... 2023-03-08 19:54:42,278 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c1edfa28825217aa75bd370b97cfc23d 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:54:42,399 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 19:54:42,690 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512311340b5b30c4c40de88cc50bc43be1686 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512311340b5b30c4c40de88cc50bc43be1686 2023-03-08 19:54:43,098 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/.tmp/family1/961d0bc99ac84fb98ed7d47610eb05dd 2023-03-08 19:54:43,103 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/.tmp/family1/961d0bc99ac84fb98ed7d47610eb05dd as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/961d0bc99ac84fb98ed7d47610eb05dd 2023-03-08 19:54:43,107 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/961d0bc99ac84fb98ed7d47610eb05dd, entries=1, sequenceid=25, filesize=5.0 K 2023-03-08 19:54:43,108 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c1edfa28825217aa75bd370b97cfc23d in 830ms, sequenceid=25, compaction requested=true 2023-03-08 19:54:43,108 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c1edfa28825217aa75bd370b97cfc23d: 2023-03-08 19:54:43,108 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:43,108 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:43,108 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:43,108 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:43,108 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-08 19:54:43,108 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-08 19:54:43,110 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:43,110 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:43,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:43,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:43,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:43,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:43,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:43,111 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:43,112 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:43,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:43,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:43,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:43,113 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-08 19:54:43,113 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6659e27a[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:43,113 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-08 19:54:43,113 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:43,114 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:43,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:43,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:43,114 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-08 19:54:43,114 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-08 19:54:43,114 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:43,114 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:43,114 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:43,116 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:43,116 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-08 19:54:43,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:43,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:43,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:43,116 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:43,116 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-08 19:54:43,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:43,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:43,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:43,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:43,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:43,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:43,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:43,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:43,118 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-08 19:54:43,118 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-08 19:54:43,118 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:43,118 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:43,118 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:43,118 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:43,120 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:43,120 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:43,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:43,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:43,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:43,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:43,120 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:43,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:43,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:43,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:43,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:43,121 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:43,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:43,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:43,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:43,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:43,122 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:43,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:43,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:43,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:43,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:43,124 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:43,124 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:43,124 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:43,124 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:43,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:43,124 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:43,125 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 19:54:43,125 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:43,124 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:43,125 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:43,125 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:43,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:43,125 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:43,125 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:43,125 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-08 19:54:43,125 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:43,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:43,125 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:43,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:43,225 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:43,226 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:43,231 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:43,233 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:43,233 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:43,233 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 19:54:43,233 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:43,233 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-08 19:54:43,233 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-08 19:54:43,233 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:43,233 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:43,235 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:43,235 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:43,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:43,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:43,235 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:43,235 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:43,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:43,235 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:43,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:43,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:43,236 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:43,236 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-08 19:54:43,236 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:43,236 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-08 19:54:43,236 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:43,236 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-08 19:54:43,236 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:43,236 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:43,236 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:43,236 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. started... 2023-03-08 19:54:43,236 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c1edfa28825217aa75bd370b97cfc23d 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:54:43,649 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010107d7b626142445f7ba3c333522ea7dd9 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2016010107d7b626142445f7ba3c333522ea7dd9 2023-03-08 19:54:44,055 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/.tmp/family1/c56d3b71dde848ce9fd4d6aab593ffc3 2023-03-08 19:54:44,060 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/.tmp/family1/c56d3b71dde848ce9fd4d6aab593ffc3 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/c56d3b71dde848ce9fd4d6aab593ffc3 2023-03-08 19:54:44,064 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/c56d3b71dde848ce9fd4d6aab593ffc3, entries=1, sequenceid=29, filesize=5.0 K 2023-03-08 19:54:44,065 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c1edfa28825217aa75bd370b97cfc23d in 829ms, sequenceid=29, compaction requested=true 2023-03-08 19:54:44,065 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c1edfa28825217aa75bd370b97cfc23d: 2023-03-08 19:54:44,065 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:44,065 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:44,065 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:44,065 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:44,065 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-08 19:54:44,065 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-08 19:54:44,067 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:44,067 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:44,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:44,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:44,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:44,067 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:44,067 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:44,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:44,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:44,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:44,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:44,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:44,068 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-08 19:54:44,069 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@65f6a021[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:44,069 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-08 19:54:44,069 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:44,070 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:44,070 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:44,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:44,070 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-08 19:54:44,070 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-08 19:54:44,070 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:44,070 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:44,070 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:44,073 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:44,073 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-08 19:54:44,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:44,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:44,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:44,073 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:44,074 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-08 19:54:44,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:44,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:44,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:44,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:44,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:44,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:44,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:44,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:44,076 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-08 19:54:44,076 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-08 19:54:44,076 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:44,076 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:44,076 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:44,076 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:44,079 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:44,079 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:44,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:44,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:44,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:44,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:44,079 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:44,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:44,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:44,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:44,080 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:44,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:44,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:44,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:44,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:44,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:44,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:44,081 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:44,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:44,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:44,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:44,083 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:44,083 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:44,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:44,083 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:44,083 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-08 19:54:44,083 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:44,083 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:44,084 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:44,083 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 19:54:44,084 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:44,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:44,084 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:44,084 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:44,084 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:44,084 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:44,084 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:44,084 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:44,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:44,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:44,184 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:44,185 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:44,189 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:44,191 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:44,191 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:44,191 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 19:54:44,191 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:44,191 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-08 19:54:44,191 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-08 19:54:44,191 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:44,191 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:44,193 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:44,193 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:44,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:44,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:44,194 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:44,194 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:44,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:44,194 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:44,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:44,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:44,194 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:44,194 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-08 19:54:44,195 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:44,195 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-08 19:54:44,195 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:44,195 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-08 19:54:44,195 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:44,195 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:44,195 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:44,195 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. started... 2023-03-08 19:54:44,195 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c1edfa28825217aa75bd370b97cfc23d 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:54:44,609 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103b7db95b261bb4ade8e138cdc88d08f5f to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103b7db95b261bb4ade8e138cdc88d08f5f 2023-03-08 19:54:45,015 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/.tmp/family1/2dfb68ccc21b4ef683cccd0c0b7cce4a 2023-03-08 19:54:45,020 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/.tmp/family1/2dfb68ccc21b4ef683cccd0c0b7cce4a as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/2dfb68ccc21b4ef683cccd0c0b7cce4a 2023-03-08 19:54:45,024 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/2dfb68ccc21b4ef683cccd0c0b7cce4a, entries=1, sequenceid=33, filesize=5.0 K 2023-03-08 19:54:45,025 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c1edfa28825217aa75bd370b97cfc23d in 830ms, sequenceid=33, compaction requested=true 2023-03-08 19:54:45,025 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c1edfa28825217aa75bd370b97cfc23d: 2023-03-08 19:54:45,025 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:45,025 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:45,025 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:45,025 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:45,025 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-08 19:54:45,025 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-08 19:54:45,027 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:45,027 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:45,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:45,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:45,027 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,027 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:45,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:45,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:45,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:45,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:45,028 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-08 19:54:45,028 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@50661783[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:45,028 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-08 19:54:45,029 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,030 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,030 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-08 19:54:45,030 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:45,030 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:45,030 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-08 19:54:45,030 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:45,032 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-08 19:54:45,032 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:45,032 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:45,032 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-08 19:54:45,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:45,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:45,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:45,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:45,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:45,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:45,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:45,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:45,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:45,034 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-08 19:54:45,034 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-08 19:54:45,034 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:45,034 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:45,034 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,034 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:45,036 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,036 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,036 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:45,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:45,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:45,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:45,036 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:45,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:45,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:45,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:45,037 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:45,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:45,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:45,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:45,040 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:45,040 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:45,040 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:45,040 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:45,040 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,040 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,040 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:45,040 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,040 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,040 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 19:54:45,040 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:45,040 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:45,040 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-08 19:54:45,040 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:45,040 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:45,040 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:45,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:45,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:45,141 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:45,142 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:45,146 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:45,148 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:45,148 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,148 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 19:54:45,148 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:45,148 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-08 19:54:45,148 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-08 19:54:45,149 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,149 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,150 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:45,150 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:45,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:45,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:45,151 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:45,151 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:45,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,151 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:45,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,152 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,152 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-08 19:54:45,152 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,152 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-08 19:54:45,152 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:45,152 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-08 19:54:45,152 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:45,152 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:45,152 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:45,152 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. started... 2023-03-08 19:54:45,153 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c1edfa28825217aa75bd370b97cfc23d 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:54:45,570 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907a45a3daae53245c5bebe68f6c3a39000 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20150907a45a3daae53245c5bebe68f6c3a39000 2023-03-08 19:54:45,976 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/.tmp/family1/fc4bd0e9fdd54a7fa22de35e6d8fa7fa 2023-03-08 19:54:45,981 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/.tmp/family1/fc4bd0e9fdd54a7fa22de35e6d8fa7fa as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/fc4bd0e9fdd54a7fa22de35e6d8fa7fa 2023-03-08 19:54:45,986 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/fc4bd0e9fdd54a7fa22de35e6d8fa7fa, entries=1, sequenceid=37, filesize=5.0 K 2023-03-08 19:54:45,986 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c1edfa28825217aa75bd370b97cfc23d in 833ms, sequenceid=37, compaction requested=true 2023-03-08 19:54:45,987 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c1edfa28825217aa75bd370b97cfc23d: 2023-03-08 19:54:45,987 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:45,987 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:45,987 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:45,987 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:45,987 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-08 19:54:45,987 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-08 19:54:45,989 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:45,989 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:45,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:45,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:45,989 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,989 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:45,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:45,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:45,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:45,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:45,991 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-08 19:54:45,991 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@298e55aa[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:45,991 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-08 19:54:45,991 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,992 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,993 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-08 19:54:45,993 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-08 19:54:45,993 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:45,993 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:45,993 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:45,994 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:45,994 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-08 19:54:45,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:45,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:45,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:45,994 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:45,994 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-08 19:54:45,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:45,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:45,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:45,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:45,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:45,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:45,997 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-08 19:54:45,997 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-08 19:54:45,997 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:45,997 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:45,997 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,997 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:45,998 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,998 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:45,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:45,999 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:45,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:45,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:45,999 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:45,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:45,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:45,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:46,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:46,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:46,000 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:46,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:46,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:46,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:46,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:46,006 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:46,006 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:46,006 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:46,006 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:46,006 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:46,006 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:46,006 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 19:54:46,006 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:46,006 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:46,006 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:46,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:46,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:46,006 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:46,007 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:46,006 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:46,007 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:46,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:46,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:46,107 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:46,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:46,112 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:46,114 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:46,114 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:46,114 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 19:54:46,114 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:46,114 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-08 19:54:46,114 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-08 19:54:46,114 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:46,114 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:46,116 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:46,116 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:46,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:46,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:46,116 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:46,116 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:46,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:46,116 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:46,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:46,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:46,117 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:46,117 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-08 19:54:46,117 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:46,117 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-08 19:54:46,117 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:46,117 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-08 19:54:46,117 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:46,117 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:46,118 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:46,118 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. started... 2023-03-08 19:54:46,118 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c1edfa28825217aa75bd370b97cfc23d 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:54:46,531 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511204a8a3b350b7344678e315c759b9cc3e6 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511204a8a3b350b7344678e315c759b9cc3e6 2023-03-08 19:54:46,937 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/.tmp/family1/21265ec245914b46ab75a78adcda7d1c 2023-03-08 19:54:46,942 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/.tmp/family1/21265ec245914b46ab75a78adcda7d1c as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/21265ec245914b46ab75a78adcda7d1c 2023-03-08 19:54:46,945 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/21265ec245914b46ab75a78adcda7d1c, entries=1, sequenceid=41, filesize=5.0 K 2023-03-08 19:54:46,946 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c1edfa28825217aa75bd370b97cfc23d in 828ms, sequenceid=41, compaction requested=true 2023-03-08 19:54:46,946 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c1edfa28825217aa75bd370b97cfc23d: 2023-03-08 19:54:46,946 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:46,946 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:46,946 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:46,946 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:46,946 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-08 19:54:46,946 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-08 19:54:46,948 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:46,948 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:46,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:46,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:46,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:46,948 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:46,948 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:46,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:46,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:46,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:46,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:46,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:46,950 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-08 19:54:46,950 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@ed38d3[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:46,950 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-08 19:54:46,950 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:46,951 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:46,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:46,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:46,951 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-08 19:54:46,951 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-08 19:54:46,951 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:46,951 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:46,951 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:46,953 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:46,953 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-08 19:54:46,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:46,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:46,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:46,953 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:46,953 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-08 19:54:46,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:46,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:46,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:46,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:46,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:46,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:46,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:46,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:46,956 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-08 19:54:46,956 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-08 19:54:46,956 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:46,956 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:46,956 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:46,956 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:46,958 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:46,958 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:46,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:46,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:46,959 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:46,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:46,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:46,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:46,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:46,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:46,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:46,959 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:46,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:46,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:46,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:46,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:46,960 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:46,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:46,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:46,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:46,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:46,962 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:46,962 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:46,962 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:46,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:46,962 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:46,963 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:46,963 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:46,963 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:46,963 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-08 19:54:46,962 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-08 19:54:46,963 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:46,962 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:46,962 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:46,963 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:46,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:46,963 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:46,963 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:46,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:46,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:47,063 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:47,065 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:47,066 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39989] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:47,067 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] procedure2.ProcedureExecutor(1029): Stored pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-08 19:54:47,068 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-08 19:54:47,069 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 19:54:47,069 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=42, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678305887069 2023-03-08 19:54:47,071 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 19:54:47,072 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-03-08 19:54:47,072 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 19:54:47,073 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] zookeeper.ReadOnlyZKClient(139): Connect 0x0279552a to 127.0.0.1:56751 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 19:54:47,077 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@743984c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 19:54:47,077 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 19:54:47,518 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103af95d0dc97cf48e5a6968a2d85a5b8ff to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103af95d0dc97cf48e5a6968a2d85a5b8ff 2023-03-08 19:54:47,519 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 19:54:47,531 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 19:54:47,532 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52820, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 19:54:47,539 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 19:54:47,541 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36658, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 19:54:47,554 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/992659dd79a143e68322290a08959e4a first=Optional[r3] last=Optional[r6] 2023-03-08 19:54:47,554 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d., hostname=jenkins-hbase4.apache.org,33367,1678305218680, seqNum=2 for row with hfile group [{family1,hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/992659dd79a143e68322290a08959e4a}] 2023-03-08 19:54:47,559 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(763): Validating hfile at hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/992659dd79a143e68322290a08959e4a for inclusion in c1edfa28825217aa75bd370b97cfc23d/family1 2023-03-08 19:54:47,561 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-08 19:54:47,561 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 19:54:47,562 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(2446): Flush status journal for c1edfa28825217aa75bd370b97cfc23d: 2023-03-08 19:54:47,562 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/992659dd79a143e68322290a08959e4a to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__o9fu9f2ce8fuugo8nilk7me6bh4peoc0glams6udm7ve1dbr4lhrsu29u0vj5917/family1/992659dd79a143e68322290a08959e4a 2023-03-08 19:54:47,565 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__o9fu9f2ce8fuugo8nilk7me6bh4peoc0glams6udm7ve1dbr4lhrsu29u0vj5917/family1/992659dd79a143e68322290a08959e4a as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/1c3ed8baeec3454882ac347dcf94c838_SeqId_44_ 2023-03-08 19:54:47,566 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(849): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__o9fu9f2ce8fuugo8nilk7me6bh4peoc0glams6udm7ve1dbr4lhrsu29u0vj5917/family1/992659dd79a143e68322290a08959e4a into c1edfa28825217aa75bd370b97cfc23d/family1 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/1c3ed8baeec3454882ac347dcf94c838_SeqId_44_ - updating store file list. 2023-03-08 19:54:47,569 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(882): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/1c3ed8baeec3454882ac347dcf94c838_SeqId_44_ into c1edfa28825217aa75bd370b97cfc23d/family1 2023-03-08 19:54:47,569 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(855): Successfully loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__o9fu9f2ce8fuugo8nilk7me6bh4peoc0glams6udm7ve1dbr4lhrsu29u0vj5917/family1/992659dd79a143e68322290a08959e4a into c1edfa28825217aa75bd370b97cfc23d/family1 (new location: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/1c3ed8baeec3454882ac347dcf94c838_SeqId_44_) 2023-03-08 19:54:47,570 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__o9fu9f2ce8fuugo8nilk7me6bh4peoc0glams6udm7ve1dbr4lhrsu29u0vj5917/family1/992659dd79a143e68322290a08959e4a 2023-03-08 19:54:47,570 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-08 19:54:47,573 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51565 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-08 19:54:47,573 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(1912): c1edfa28825217aa75bd370b97cfc23d/family1 is initiating minor compaction 2023-03-08 19:54:47,573 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@23084b44; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 19:54:47,573 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of c1edfa28825217aa75bd370b97cfc23d/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:47,573 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/b364ac349c97413e8208c15ad6755201, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/ccd761b0780f45158f9dd1b14525b38d, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/4041e25692384f6ab2e15ec6bb6a8e59, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/c87aee1ea61445c9b1270a9d4ce5ec7a, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/961d0bc99ac84fb98ed7d47610eb05dd, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/c56d3b71dde848ce9fd4d6aab593ffc3, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/2dfb68ccc21b4ef683cccd0c0b7cce4a, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/fc4bd0e9fdd54a7fa22de35e6d8fa7fa, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/21265ec245914b46ab75a78adcda7d1c, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/1c3ed8baeec3454882ac347dcf94c838_SeqId_44_] into tmpdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/.tmp, totalSize=50.4 K 2023-03-08 19:54:47,573 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting b364ac349c97413e8208c15ad6755201, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-08 19:54:47,574 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting ccd761b0780f45158f9dd1b14525b38d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-08 19:54:47,574 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 4041e25692384f6ab2e15ec6bb6a8e59, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-08 19:54:47,574 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting c87aee1ea61445c9b1270a9d4ce5ec7a, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-08 19:54:47,574 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 961d0bc99ac84fb98ed7d47610eb05dd, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-08 19:54:47,574 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting c56d3b71dde848ce9fd4d6aab593ffc3, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-08 19:54:47,574 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 2dfb68ccc21b4ef683cccd0c0b7cce4a, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-08 19:54:47,574 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting fc4bd0e9fdd54a7fa22de35e6d8fa7fa, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-08 19:54:47,574 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 21265ec245914b46ab75a78adcda7d1c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-08 19:54:47,574 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 1c3ed8baeec3454882ac347dcf94c838_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-08 19:54:47,576 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 19:54:47,576 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:54:47,578 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151228d00ee7dcf570452aa657098dcd7c9ce3 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151228d00ee7dcf570452aa657098dcd7c9ce3 2023-03-08 19:54:47,580 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512311340b5b30c4c40de88cc50bc43be1686 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512311340b5b30c4c40de88cc50bc43be1686 2023-03-08 19:54:47,581 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2016010107d7b626142445f7ba3c333522ea7dd9 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2016010107d7b626142445f7ba3c333522ea7dd9 2023-03-08 19:54:47,582 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103b7db95b261bb4ade8e138cdc88d08f5f to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103b7db95b261bb4ade8e138cdc88d08f5f 2023-03-08 19:54:47,582 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 4 expired mob files are deleted 2023-03-08 19:54:47,582 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-08 19:54:47,582 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 19:54:47,583 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 19:54:47,583 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 19:54:47,583 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 19:54:47,583 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 19:54:47,583 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 19:54:47,583 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 19:54:47,583 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 19:54:47,602 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] throttle.PressureAwareThroughputController(145): c1edfa28825217aa75bd370b97cfc23d#family1#compaction#61 average throughput is 0.79 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 19:54:48,037 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512057db13fb11e7d41b2b22091d9bfefb4bf to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512057db13fb11e7d41b2b22091d9bfefb4bf 2023-03-08 19:54:48,038 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 19:54:48,070 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7173c6a8af7b485b836676f02617ebe4 first=Optional[r00] last=Optional[r2] 2023-03-08 19:54:48,070 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d., hostname=jenkins-hbase4.apache.org,33367,1678305218680, seqNum=2 for row with hfile group [{family1,hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7173c6a8af7b485b836676f02617ebe4}] 2023-03-08 19:54:48,072 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(763): Validating hfile at hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7173c6a8af7b485b836676f02617ebe4 for inclusion in c1edfa28825217aa75bd370b97cfc23d/family1 2023-03-08 19:54:48,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-08 19:54:48,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 19:54:48,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(2446): Flush status journal for c1edfa28825217aa75bd370b97cfc23d: 2023-03-08 19:54:48,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7173c6a8af7b485b836676f02617ebe4 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__gd5qt8s3i34kmgvik10iu41b3g6k7ch0ekqstocc35fm7f86qe3gu071hn29tc7o/family1/7173c6a8af7b485b836676f02617ebe4 2023-03-08 19:54:48,079 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__gd5qt8s3i34kmgvik10iu41b3g6k7ch0ekqstocc35fm7f86qe3gu071hn29tc7o/family1/7173c6a8af7b485b836676f02617ebe4 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/4508fc25ada34a2cad54586348d5571c_SeqId_46_ 2023-03-08 19:54:48,079 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(849): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__gd5qt8s3i34kmgvik10iu41b3g6k7ch0ekqstocc35fm7f86qe3gu071hn29tc7o/family1/7173c6a8af7b485b836676f02617ebe4 into c1edfa28825217aa75bd370b97cfc23d/family1 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/4508fc25ada34a2cad54586348d5571c_SeqId_46_ - updating store file list. 2023-03-08 19:54:48,083 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(882): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/4508fc25ada34a2cad54586348d5571c_SeqId_46_ into c1edfa28825217aa75bd370b97cfc23d/family1 2023-03-08 19:54:48,083 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(855): Successfully loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__gd5qt8s3i34kmgvik10iu41b3g6k7ch0ekqstocc35fm7f86qe3gu071hn29tc7o/family1/7173c6a8af7b485b836676f02617ebe4 into c1edfa28825217aa75bd370b97cfc23d/family1 (new location: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/4508fc25ada34a2cad54586348d5571c_SeqId_46_) 2023-03-08 19:54:48,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__gd5qt8s3i34kmgvik10iu41b3g6k7ch0ekqstocc35fm7f86qe3gu071hn29tc7o/family1/7173c6a8af7b485b836676f02617ebe4 2023-03-08 19:54:48,085 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 19:54:48,086 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:54:48,088 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130c7821063676e45e4bf7bbc9a7558f089 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130c7821063676e45e4bf7bbc9a7558f089 2023-03-08 19:54:48,089 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120179a651cb293043829ea2b480da16305d to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120179a651cb293043829ea2b480da16305d 2023-03-08 19:54:48,090 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512054f41dbf198ed4eb9a948421f765d631b to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512054f41dbf198ed4eb9a948421f765d631b 2023-03-08 19:54:48,090 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 3 expired mob files are deleted 2023-03-08 19:54:48,090 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-08 19:54:48,090 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 19:54:48,090 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 19:54:48,090 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 19:54:48,091 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=42, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-08 19:54:48,093 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=42, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 1.0250 sec 2023-03-08 19:54:48,182 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-08 19:54:48,183 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure2.ProcedureExecutor(1029): Stored pid=43, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:48,420 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/.tmp/family1/f4a8bb72452a4aeb89da9e17827a1674 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/f4a8bb72452a4aeb89da9e17827a1674 2023-03-08 19:54:48,424 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in c1edfa28825217aa75bd370b97cfc23d/family1 of c1edfa28825217aa75bd370b97cfc23d into f4a8bb72452a4aeb89da9e17827a1674(size=6.8 K), total size for store is 390.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 19:54:48,424 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for c1edfa28825217aa75bd370b97cfc23d: 2023-03-08 19:54:48,424 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d., storeName=family1, fileCount=10, fileSize=50.4 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1678305287573; duration=0sec 2023-03-08 19:54:48,424 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 19:54:48,524 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/b364ac349c97413e8208c15ad6755201, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/ccd761b0780f45158f9dd1b14525b38d, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/4041e25692384f6ab2e15ec6bb6a8e59, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/c87aee1ea61445c9b1270a9d4ce5ec7a, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/961d0bc99ac84fb98ed7d47610eb05dd, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/c56d3b71dde848ce9fd4d6aab593ffc3, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/2dfb68ccc21b4ef683cccd0c0b7cce4a, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/fc4bd0e9fdd54a7fa22de35e6d8fa7fa, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/21265ec245914b46ab75a78adcda7d1c, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/1c3ed8baeec3454882ac347dcf94c838_SeqId_44_] to archive 2023-03-08 19:54:48,525 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:54:48,527 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/b364ac349c97413e8208c15ad6755201 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/b364ac349c97413e8208c15ad6755201 2023-03-08 19:54:48,528 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/ccd761b0780f45158f9dd1b14525b38d to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/ccd761b0780f45158f9dd1b14525b38d 2023-03-08 19:54:48,529 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/4041e25692384f6ab2e15ec6bb6a8e59 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/4041e25692384f6ab2e15ec6bb6a8e59 2023-03-08 19:54:48,530 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/c87aee1ea61445c9b1270a9d4ce5ec7a to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/c87aee1ea61445c9b1270a9d4ce5ec7a 2023-03-08 19:54:48,531 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/961d0bc99ac84fb98ed7d47610eb05dd to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/961d0bc99ac84fb98ed7d47610eb05dd 2023-03-08 19:54:48,532 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/c56d3b71dde848ce9fd4d6aab593ffc3 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/c56d3b71dde848ce9fd4d6aab593ffc3 2023-03-08 19:54:48,534 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/2dfb68ccc21b4ef683cccd0c0b7cce4a to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/2dfb68ccc21b4ef683cccd0c0b7cce4a 2023-03-08 19:54:48,535 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/fc4bd0e9fdd54a7fa22de35e6d8fa7fa to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/fc4bd0e9fdd54a7fa22de35e6d8fa7fa 2023-03-08 19:54:48,536 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/21265ec245914b46ab75a78adcda7d1c to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/21265ec245914b46ab75a78adcda7d1c 2023-03-08 19:54:48,537 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/1c3ed8baeec3454882ac347dcf94c838_SeqId_44_ to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/1c3ed8baeec3454882ac347dcf94c838_SeqId_44_ 2023-03-08 19:54:48,593 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-03-08 19:54:48,594 DEBUG [PEWorker-2] util.FSTableDescriptors(519): Deleted hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-03-08 19:54:48,594 INFO [PEWorker-2] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-03-08 19:54:48,596 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=44, ppid=43, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-03-08 19:54:48,598 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=c1edfa28825217aa75bd370b97cfc23d, REOPEN/MOVE}] 2023-03-08 19:54:48,599 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=c1edfa28825217aa75bd370b97cfc23d, REOPEN/MOVE 2023-03-08 19:54:48,599 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=c1edfa28825217aa75bd370b97cfc23d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:48,600 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678305288599"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678305288599"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678305288599"}]},"ts":"1678305288599"} 2023-03-08 19:54:48,601 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure c1edfa28825217aa75bd370b97cfc23d, server=jenkins-hbase4.apache.org,33367,1678305218680}] 2023-03-08 19:54:48,753 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close c1edfa28825217aa75bd370b97cfc23d 2023-03-08 19:54:48,753 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing c1edfa28825217aa75bd370b97cfc23d, disabling compactions & flushes 2023-03-08 19:54:48,753 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:48,753 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:48,753 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. after waiting 0 ms 2023-03-08 19:54:48,753 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:48,758 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/recovered.edits/49.seqid, newMaxSeqId=49, maxSeqId=1 2023-03-08 19:54:48,759 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:48,759 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for c1edfa28825217aa75bd370b97cfc23d: 2023-03-08 19:54:48,759 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: c1edfa28825217aa75bd370b97cfc23d to self. 2023-03-08 19:54:48,761 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed c1edfa28825217aa75bd370b97cfc23d 2023-03-08 19:54:48,761 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=c1edfa28825217aa75bd370b97cfc23d, regionState=CLOSED 2023-03-08 19:54:48,761 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678305288761"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678305288761"}]},"ts":"1678305288761"} 2023-03-08 19:54:48,764 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-03-08 19:54:48,764 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure c1edfa28825217aa75bd370b97cfc23d, server=jenkins-hbase4.apache.org,33367,1678305218680 in 162 msec 2023-03-08 19:54:48,765 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=c1edfa28825217aa75bd370b97cfc23d, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,33367,1678305218680; forceNewPlan=false, retain=true 2023-03-08 19:54:48,915 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=c1edfa28825217aa75bd370b97cfc23d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:48,915 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678305288915"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678305288915"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678305288915"}]},"ts":"1678305288915"} 2023-03-08 19:54:48,917 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=47, ppid=45, state=RUNNABLE; OpenRegionProcedure c1edfa28825217aa75bd370b97cfc23d, server=jenkins-hbase4.apache.org,33367,1678305218680}] 2023-03-08 19:54:49,072 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:49,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c1edfa28825217aa75bd370b97cfc23d, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d.', STARTKEY => '', ENDKEY => ''} 2023-03-08 19:54:49,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly c1edfa28825217aa75bd370b97cfc23d 2023-03-08 19:54:49,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:54:49,073 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for c1edfa28825217aa75bd370b97cfc23d 2023-03-08 19:54:49,073 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for c1edfa28825217aa75bd370b97cfc23d 2023-03-08 19:54:49,074 INFO [StoreOpener-c1edfa28825217aa75bd370b97cfc23d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region c1edfa28825217aa75bd370b97cfc23d 2023-03-08 19:54:49,074 DEBUG [StoreOpener-c1edfa28825217aa75bd370b97cfc23d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1 2023-03-08 19:54:49,075 DEBUG [StoreOpener-c1edfa28825217aa75bd370b97cfc23d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1 2023-03-08 19:54:49,075 INFO [StoreOpener-c1edfa28825217aa75bd370b97cfc23d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c1edfa28825217aa75bd370b97cfc23d columnFamilyName family1 2023-03-08 19:54:49,080 DEBUG [StoreOpener-c1edfa28825217aa75bd370b97cfc23d-1] regionserver.HStore(539): loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/4508fc25ada34a2cad54586348d5571c_SeqId_46_ 2023-03-08 19:54:49,084 DEBUG [StoreOpener-c1edfa28825217aa75bd370b97cfc23d-1] regionserver.HStore(539): loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/9aadb18f51df4e97a73e6367195c7aa3 2023-03-08 19:54:49,087 DEBUG [StoreOpener-c1edfa28825217aa75bd370b97cfc23d-1] regionserver.HStore(539): loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/f4a8bb72452a4aeb89da9e17827a1674 2023-03-08 19:54:49,087 INFO [StoreOpener-c1edfa28825217aa75bd370b97cfc23d-1] regionserver.HStore(310): Store=c1edfa28825217aa75bd370b97cfc23d/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:54:49,088 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d 2023-03-08 19:54:49,089 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d 2023-03-08 19:54:49,092 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for c1edfa28825217aa75bd370b97cfc23d 2023-03-08 19:54:49,093 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened c1edfa28825217aa75bd370b97cfc23d; next sequenceid=50; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62163444, jitterRate=-0.07369250059127808}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 19:54:49,093 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for c1edfa28825217aa75bd370b97cfc23d: 2023-03-08 19:54:49,093 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d., pid=47, masterSystemTime=1678305289069 2023-03-08 19:54:49,094 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 19:54:49,094 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-08 19:54:49,096 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 399433 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-08 19:54:49,096 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1912): c1edfa28825217aa75bd370b97cfc23d/family1 is initiating minor compaction (all files) 2023-03-08 19:54:49,096 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of c1edfa28825217aa75bd370b97cfc23d/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:49,096 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/9aadb18f51df4e97a73e6367195c7aa3, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/f4a8bb72452a4aeb89da9e17827a1674, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/4508fc25ada34a2cad54586348d5571c_SeqId_46_] into tmpdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/.tmp, totalSize=390.1 K 2023-03-08 19:54:49,096 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:49,096 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:49,097 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=c1edfa28825217aa75bd370b97cfc23d, regionState=OPEN, openSeqNum=50, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:49,097 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 9aadb18f51df4e97a73e6367195c7aa3, keycount=1000, bloomtype=ROW, size=191.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878843488 2023-03-08 19:54:49,097 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678305289097"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678305289097"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678305289097"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678305289097"}]},"ts":"1678305289097"} 2023-03-08 19:54:49,097 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting f4a8bb72452a4aeb89da9e17827a1674, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621243488 2023-03-08 19:54:49,098 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 4508fc25ada34a2cad54586348d5571c_SeqId_46_, keycount=1002, bloomtype=ROW, size=191.8 K, encoding=NONE, compression=NONE, seqNum=47, earliestPutTs=1448878843488 2023-03-08 19:54:49,100 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=47, resume processing ppid=45 2023-03-08 19:54:49,100 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=47, ppid=45, state=SUCCESS; OpenRegionProcedure c1edfa28825217aa75bd370b97cfc23d, server=jenkins-hbase4.apache.org,33367,1678305218680 in 181 msec 2023-03-08 19:54:49,101 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-03-08 19:54:49,101 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=c1edfa28825217aa75bd370b97cfc23d, REOPEN/MOVE in 502 msec 2023-03-08 19:54:49,104 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=44, resume processing ppid=43 2023-03-08 19:54:49,104 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=44, ppid=43, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 506 msec 2023-03-08 19:54:49,106 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 923 msec 2023-03-08 19:54:49,107 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=43 2023-03-08 19:54:49,108 INFO [Listener at localhost/34049] compactions.TestMobCompactor(278): alter status finished 2023-03-08 19:54:49,112 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39989] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:49,113 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] throttle.PressureAwareThroughputController(145): c1edfa28825217aa75bd370b97cfc23d#family1#compaction#62 average throughput is 25.80 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 19:54:49,114 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] procedure2.ProcedureExecutor(1029): Stored pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-08 19:54:49,114 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-08 19:54:49,115 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 19:54:49,115 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=48, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678305889115 2023-03-08 19:54:49,116 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 19:54:49,116 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-03-08 19:54:49,117 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 19:54:49,117 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] zookeeper.ReadOnlyZKClient(139): Connect 0x6ea2c454 to 127.0.0.1:56751 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 19:54:49,133 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@75b8a034, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 19:54:49,133 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 19:54:49,133 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 19:54:49,133 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 19:54:49,133 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 19:54:49,133 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 19:54:49,965 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128d88ad74cb11a4528b2e06132fdfbe883 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128d88ad74cb11a4528b2e06132fdfbe883 2023-03-08 19:54:49,965 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 19:54:49,978 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 19:54:49,979 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52822, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 19:54:49,986 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 19:54:49,987 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36674, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 19:54:49,996 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/d169cb55334b4d42a4a3cd643dd00901 first=Optional[r05] last=Optional[r8] 2023-03-08 19:54:49,997 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d., hostname=jenkins-hbase4.apache.org,33367,1678305218680, seqNum=50 for row with hfile group [{family1,hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/d169cb55334b4d42a4a3cd643dd00901}] 2023-03-08 19:54:50,003 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(763): Validating hfile at hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/d169cb55334b4d42a4a3cd643dd00901 for inclusion in c1edfa28825217aa75bd370b97cfc23d/family1 2023-03-08 19:54:50,006 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-03-08 19:54:50,006 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 19:54:50,007 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(2446): Flush status journal for c1edfa28825217aa75bd370b97cfc23d: 2023-03-08 19:54:50,008 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/d169cb55334b4d42a4a3cd643dd00901 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__4uqga0159tqdl78kae5p8voefidmmqi3ijfilrkn4gp8tk84cko13qrc53vve2u8/family1/d169cb55334b4d42a4a3cd643dd00901 2023-03-08 19:54:50,011 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__4uqga0159tqdl78kae5p8voefidmmqi3ijfilrkn4gp8tk84cko13qrc53vve2u8/family1/d169cb55334b4d42a4a3cd643dd00901 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/26c5666adf3346c5b63b15d68fd058d7_SeqId_52_ 2023-03-08 19:54:50,012 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(849): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__4uqga0159tqdl78kae5p8voefidmmqi3ijfilrkn4gp8tk84cko13qrc53vve2u8/family1/d169cb55334b4d42a4a3cd643dd00901 into c1edfa28825217aa75bd370b97cfc23d/family1 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/26c5666adf3346c5b63b15d68fd058d7_SeqId_52_ - updating store file list. 2023-03-08 19:54:50,016 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(882): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/26c5666adf3346c5b63b15d68fd058d7_SeqId_52_ into c1edfa28825217aa75bd370b97cfc23d/family1 2023-03-08 19:54:50,016 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(855): Successfully loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__4uqga0159tqdl78kae5p8voefidmmqi3ijfilrkn4gp8tk84cko13qrc53vve2u8/family1/d169cb55334b4d42a4a3cd643dd00901 into c1edfa28825217aa75bd370b97cfc23d/family1 (new location: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/26c5666adf3346c5b63b15d68fd058d7_SeqId_52_) 2023-03-08 19:54:50,016 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__4uqga0159tqdl78kae5p8voefidmmqi3ijfilrkn4gp8tk84cko13qrc53vve2u8/family1/d169cb55334b4d42a4a3cd643dd00901 2023-03-08 19:54:50,018 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 19:54:50,019 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:54:50,021 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511204a8a3b350b7344678e315c759b9cc3e6 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511204a8a3b350b7344678e315c759b9cc3e6 2023-03-08 19:54:50,022 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128b3b1e64cd21b4c7ea508978d3290ffc6 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128b3b1e64cd21b4c7ea508978d3290ffc6 2023-03-08 19:54:50,022 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2 expired mob files are deleted 2023-03-08 19:54:50,022 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-08 19:54:50,023 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 19:54:50,023 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 19:54:50,023 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 19:54:50,023 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 19:54:50,023 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 19:54:50,024 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=48, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-08 19:54:50,026 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=48, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 911 msec 2023-03-08 19:54:50,079 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-08 19:54:50,080 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:50,335 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/.tmp/family1/5d079655fe74469f9e56eea3f9003673 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/5d079655fe74469f9e56eea3f9003673 2023-03-08 19:54:50,339 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in c1edfa28825217aa75bd370b97cfc23d/family1 of c1edfa28825217aa75bd370b97cfc23d into 5d079655fe74469f9e56eea3f9003673(size=193.0 K), total size for store is 198.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 19:54:50,339 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for c1edfa28825217aa75bd370b97cfc23d: 2023-03-08 19:54:50,339 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d., storeName=c1edfa28825217aa75bd370b97cfc23d/family1, priority=13, startTime=1678305289094; duration=1sec 2023-03-08 19:54:50,340 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 19:54:50,420 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/9aadb18f51df4e97a73e6367195c7aa3, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/f4a8bb72452a4aeb89da9e17827a1674, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/4508fc25ada34a2cad54586348d5571c_SeqId_46_] to archive 2023-03-08 19:54:50,421 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:54:50,422 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/9aadb18f51df4e97a73e6367195c7aa3 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/9aadb18f51df4e97a73e6367195c7aa3 2023-03-08 19:54:50,424 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/f4a8bb72452a4aeb89da9e17827a1674 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/f4a8bb72452a4aeb89da9e17827a1674 2023-03-08 19:54:50,425 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/4508fc25ada34a2cad54586348d5571c_SeqId_46_ to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/4508fc25ada34a2cad54586348d5571c_SeqId_46_ 2023-03-08 19:54:50,490 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-03-08 19:54:50,491 DEBUG [PEWorker-3] util.FSTableDescriptors(519): Deleted hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-03-08 19:54:50,491 INFO [PEWorker-3] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-03-08 19:54:50,493 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-03-08 19:54:50,496 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=c1edfa28825217aa75bd370b97cfc23d, REOPEN/MOVE}] 2023-03-08 19:54:50,497 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=c1edfa28825217aa75bd370b97cfc23d, REOPEN/MOVE 2023-03-08 19:54:50,498 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=c1edfa28825217aa75bd370b97cfc23d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:50,498 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678305290498"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678305290498"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678305290498"}]},"ts":"1678305290498"} 2023-03-08 19:54:50,499 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=52, ppid=51, state=RUNNABLE; CloseRegionProcedure c1edfa28825217aa75bd370b97cfc23d, server=jenkins-hbase4.apache.org,33367,1678305218680}] 2023-03-08 19:54:50,651 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close c1edfa28825217aa75bd370b97cfc23d 2023-03-08 19:54:50,651 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing c1edfa28825217aa75bd370b97cfc23d, disabling compactions & flushes 2023-03-08 19:54:50,651 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:50,651 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:50,651 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. after waiting 0 ms 2023-03-08 19:54:50,652 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:50,658 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=49 2023-03-08 19:54:50,658 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:50,658 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for c1edfa28825217aa75bd370b97cfc23d: 2023-03-08 19:54:50,658 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: c1edfa28825217aa75bd370b97cfc23d to self. 2023-03-08 19:54:50,660 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed c1edfa28825217aa75bd370b97cfc23d 2023-03-08 19:54:50,660 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=c1edfa28825217aa75bd370b97cfc23d, regionState=CLOSED 2023-03-08 19:54:50,660 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678305290660"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678305290660"}]},"ts":"1678305290660"} 2023-03-08 19:54:50,663 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=52, resume processing ppid=51 2023-03-08 19:54:50,663 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=52, ppid=51, state=SUCCESS; CloseRegionProcedure c1edfa28825217aa75bd370b97cfc23d, server=jenkins-hbase4.apache.org,33367,1678305218680 in 163 msec 2023-03-08 19:54:50,663 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=c1edfa28825217aa75bd370b97cfc23d, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,33367,1678305218680; forceNewPlan=false, retain=true 2023-03-08 19:54:50,814 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=c1edfa28825217aa75bd370b97cfc23d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:50,814 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678305290814"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678305290814"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678305290814"}]},"ts":"1678305290814"} 2023-03-08 19:54:50,816 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=53, ppid=51, state=RUNNABLE; OpenRegionProcedure c1edfa28825217aa75bd370b97cfc23d, server=jenkins-hbase4.apache.org,33367,1678305218680}] 2023-03-08 19:54:50,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:50,971 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c1edfa28825217aa75bd370b97cfc23d, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d.', STARTKEY => '', ENDKEY => ''} 2023-03-08 19:54:50,971 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly c1edfa28825217aa75bd370b97cfc23d 2023-03-08 19:54:50,971 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:54:50,971 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for c1edfa28825217aa75bd370b97cfc23d 2023-03-08 19:54:50,971 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for c1edfa28825217aa75bd370b97cfc23d 2023-03-08 19:54:50,973 INFO [StoreOpener-c1edfa28825217aa75bd370b97cfc23d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region c1edfa28825217aa75bd370b97cfc23d 2023-03-08 19:54:50,974 DEBUG [StoreOpener-c1edfa28825217aa75bd370b97cfc23d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1 2023-03-08 19:54:50,974 DEBUG [StoreOpener-c1edfa28825217aa75bd370b97cfc23d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1 2023-03-08 19:54:50,974 INFO [StoreOpener-c1edfa28825217aa75bd370b97cfc23d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c1edfa28825217aa75bd370b97cfc23d columnFamilyName family1 2023-03-08 19:54:50,979 DEBUG [StoreOpener-c1edfa28825217aa75bd370b97cfc23d-1] regionserver.HStore(539): loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/26c5666adf3346c5b63b15d68fd058d7_SeqId_52_ 2023-03-08 19:54:50,982 DEBUG [StoreOpener-c1edfa28825217aa75bd370b97cfc23d-1] regionserver.HStore(539): loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/family1/5d079655fe74469f9e56eea3f9003673 2023-03-08 19:54:50,982 INFO [StoreOpener-c1edfa28825217aa75bd370b97cfc23d-1] regionserver.HStore(310): Store=c1edfa28825217aa75bd370b97cfc23d/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:54:50,983 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d 2023-03-08 19:54:50,984 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d 2023-03-08 19:54:50,987 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for c1edfa28825217aa75bd370b97cfc23d 2023-03-08 19:54:50,988 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened c1edfa28825217aa75bd370b97cfc23d; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60365088, jitterRate=-0.10049009323120117}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 19:54:50,988 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for c1edfa28825217aa75bd370b97cfc23d: 2023-03-08 19:54:50,989 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d., pid=53, masterSystemTime=1678305290968 2023-03-08 19:54:50,990 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:50,990 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:54:50,991 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=c1edfa28825217aa75bd370b97cfc23d, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:50,991 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678305290991"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678305290991"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678305290991"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678305290991"}]},"ts":"1678305290991"} 2023-03-08 19:54:50,994 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=53, resume processing ppid=51 2023-03-08 19:54:50,994 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=53, ppid=51, state=SUCCESS; OpenRegionProcedure c1edfa28825217aa75bd370b97cfc23d, server=jenkins-hbase4.apache.org,33367,1678305218680 in 176 msec 2023-03-08 19:54:50,995 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-03-08 19:54:50,996 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=c1edfa28825217aa75bd370b97cfc23d, REOPEN/MOVE in 498 msec 2023-03-08 19:54:50,998 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-03-08 19:54:50,998 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 503 msec 2023-03-08 19:54:51,000 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 920 msec 2023-03-08 19:54:51,001 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-03-08 19:54:51,003 INFO [Listener at localhost/34049] compactions.TestMobCompactor(278): alter status finished 2023-03-08 19:54:51,004 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39989] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-08 19:54:51,005 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] procedure2.ProcedureExecutor(1029): Stored pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-08 19:54:51,005 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-08 19:54:51,006 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 19:54:51,006 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=54, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678305891006 2023-03-08 19:54:51,007 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 19:54:51,007 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-03-08 19:54:51,008 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 19:54:51,008 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] zookeeper.ReadOnlyZKClient(139): Connect 0x511c347f to 127.0.0.1:56751 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 19:54:51,013 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@61159768, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 19:54:51,013 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 19:54:51,013 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 19:54:51,013 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 19:54:51,013 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 19:54:51,014 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 19:54:51,014 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 19:54:51,014 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 19:54:51,014 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-08 19:54:51,014 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-08 19:54:51,014 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 19:54:51,014 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=54, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-08 19:54:51,016 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=54, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 10 msec 2023-03-08 19:54:51,080 INFO [Listener at localhost/34049] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=244 (was 256), OpenFileDescriptor=613 (was 610) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=181 (was 204), ProcessCount=170 (was 170), AvailableMemoryMB=8346 (was 8350) 2023-03-08 19:54:51,093 INFO [Listener at localhost/34049] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=243, OpenFileDescriptor=613, MaxFileDescriptor=60000, SystemLoadAverage=181, ProcessCount=170, AvailableMemoryMB=8345 2023-03-08 19:54:51,095 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-08 19:54:51,096 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure2.ProcedureExecutor(1029): Stored pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:51,098 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 19:54:51,098 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 19:54:51,099 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily" procId is: 55 2023-03-08 19:54:51,100 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73 2023-03-08 19:54:51,100 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-08 19:54:51,101 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73 empty. 2023-03-08 19:54:51,101 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73 2023-03-08 19:54:51,101 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily regions 2023-03-08 19:54:51,201 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-08 19:54:51,403 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-08 19:54:51,509 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-03-08 19:54:51,511 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(7675): creating {ENCODED => a526a5895c0e84ca75dee0352c5e6b73, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp 2023-03-08 19:54:51,517 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:54:51,518 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1604): Closing a526a5895c0e84ca75dee0352c5e6b73, disabling compactions & flushes 2023-03-08 19:54:51,518 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:54:51,518 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:54:51,518 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. after waiting 0 ms 2023-03-08 19:54:51,518 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:54:51,518 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:54:51,518 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1558): Region close journal for a526a5895c0e84ca75dee0352c5e6b73: 2023-03-08 19:54:51,520 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 19:54:51,521 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678305291520"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678305291520"}]},"ts":"1678305291520"} 2023-03-08 19:54:51,522 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-08 19:54:51,522 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 19:54:51,523 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678305291523"}]},"ts":"1678305291523"} 2023-03-08 19:54:51,523 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLING in hbase:meta 2023-03-08 19:54:51,527 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=a526a5895c0e84ca75dee0352c5e6b73, ASSIGN}] 2023-03-08 19:54:51,528 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=a526a5895c0e84ca75dee0352c5e6b73, ASSIGN 2023-03-08 19:54:51,529 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=a526a5895c0e84ca75dee0352c5e6b73, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33367,1678305218680; forceNewPlan=false, retain=false 2023-03-08 19:54:51,680 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=a526a5895c0e84ca75dee0352c5e6b73, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:51,680 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678305291680"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678305291680"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678305291680"}]},"ts":"1678305291680"} 2023-03-08 19:54:51,682 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE; OpenRegionProcedure a526a5895c0e84ca75dee0352c5e6b73, server=jenkins-hbase4.apache.org,33367,1678305218680}] 2023-03-08 19:54:51,704 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-08 19:54:51,836 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:54:51,836 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a526a5895c0e84ca75dee0352c5e6b73, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73.', STARTKEY => '', ENDKEY => ''} 2023-03-08 19:54:51,836 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily a526a5895c0e84ca75dee0352c5e6b73 2023-03-08 19:54:51,836 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:54:51,836 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a526a5895c0e84ca75dee0352c5e6b73 2023-03-08 19:54:51,837 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a526a5895c0e84ca75dee0352c5e6b73 2023-03-08 19:54:51,838 INFO [StoreOpener-a526a5895c0e84ca75dee0352c5e6b73-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a526a5895c0e84ca75dee0352c5e6b73 2023-03-08 19:54:51,839 DEBUG [StoreOpener-a526a5895c0e84ca75dee0352c5e6b73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1 2023-03-08 19:54:51,839 DEBUG [StoreOpener-a526a5895c0e84ca75dee0352c5e6b73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1 2023-03-08 19:54:51,839 INFO [StoreOpener-a526a5895c0e84ca75dee0352c5e6b73-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a526a5895c0e84ca75dee0352c5e6b73 columnFamilyName family1 2023-03-08 19:54:51,841 INFO [StoreOpener-a526a5895c0e84ca75dee0352c5e6b73-1] regionserver.HStore(310): Store=a526a5895c0e84ca75dee0352c5e6b73/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:54:51,842 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73 2023-03-08 19:54:51,842 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73 2023-03-08 19:54:51,844 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a526a5895c0e84ca75dee0352c5e6b73 2023-03-08 19:54:51,846 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 19:54:51,846 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a526a5895c0e84ca75dee0352c5e6b73; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66060241, jitterRate=-0.01562570035457611}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 19:54:51,846 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a526a5895c0e84ca75dee0352c5e6b73: 2023-03-08 19:54:51,847 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73., pid=57, masterSystemTime=1678305291833 2023-03-08 19:54:51,848 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:54:51,849 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:54:51,849 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=a526a5895c0e84ca75dee0352c5e6b73, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:51,849 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678305291849"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678305291849"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678305291849"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678305291849"}]},"ts":"1678305291849"} 2023-03-08 19:54:51,851 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-03-08 19:54:51,852 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; OpenRegionProcedure a526a5895c0e84ca75dee0352c5e6b73, server=jenkins-hbase4.apache.org,33367,1678305218680 in 168 msec 2023-03-08 19:54:51,853 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=56, resume processing ppid=55 2023-03-08 19:54:51,853 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=56, ppid=55, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=a526a5895c0e84ca75dee0352c5e6b73, ASSIGN in 325 msec 2023-03-08 19:54:51,853 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 19:54:51,853 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678305291853"}]},"ts":"1678305291853"} 2023-03-08 19:54:51,854 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLED in hbase:meta 2023-03-08 19:54:51,858 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 19:54:51,859 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=55, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 763 msec 2023-03-08 19:54:52,205 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-08 19:54:52,205 INFO [Listener at localhost/34049] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, procId: 55 completed 2023-03-08 19:54:52,231 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:52,233 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:52,233 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 19:54:52,233 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:52,233 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-08 19:54:52,233 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-08 19:54:52,234 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:52,234 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:52,236 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:52,236 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:52,236 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:52,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:52,236 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:52,236 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:52,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:52,237 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:52,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:52,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:52,237 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:52,237 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:52,238 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-08 19:54:52,238 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:52,238 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-08 19:54:52,238 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:52,238 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:52,238 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:54:52,238 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. started... 2023-03-08 19:54:52,239 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a526a5895c0e84ca75dee0352c5e6b73 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-08 19:54:52,664 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113037786975b822441483b3eea546c3754d to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015113037786975b822441483b3eea546c3754d 2023-03-08 19:54:53,069 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/.tmp/family1/ae6f273aeef4415585d3c817d2e83672 2023-03-08 19:54:53,074 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/.tmp/family1/ae6f273aeef4415585d3c817d2e83672 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/ae6f273aeef4415585d3c817d2e83672 2023-03-08 19:54:53,079 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/ae6f273aeef4415585d3c817d2e83672, entries=1000, sequenceid=5, filesize=190.5 K 2023-03-08 19:54:53,080 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for a526a5895c0e84ca75dee0352c5e6b73 in 841ms, sequenceid=5, compaction requested=false 2023-03-08 19:54:53,080 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 19:54:53,080 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a526a5895c0e84ca75dee0352c5e6b73: 2023-03-08 19:54:53,080 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:54:53,080 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:53,080 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:53,080 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:53,081 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-08 19:54:53,081 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-08 19:54:53,082 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:53,082 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:53,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:53,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:53,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:53,083 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:53,083 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:53,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:53,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:53,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:53,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:53,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:53,084 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-08 19:54:53,084 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@504a7747[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:53,084 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-08 19:54:53,084 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:53,085 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:53,086 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:53,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:53,086 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-08 19:54:53,086 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-08 19:54:53,086 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:53,086 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:53,086 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:53,088 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:53,088 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-08 19:54:53,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:53,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:53,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:53,088 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:53,088 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-08 19:54:53,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:53,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:53,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:53,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:53,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:53,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:53,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:53,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:53,090 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-08 19:54:53,090 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-08 19:54:53,090 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:53,090 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:53,090 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:53,090 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:53,092 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:53,092 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:53,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:53,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:53,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:53,092 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:53,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:53,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:53,092 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:53,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:53,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:53,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:53,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:53,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:53,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:53,093 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:53,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:53,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:53,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:53,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:53,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:53,098 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:53,098 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:53,098 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:53,098 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:53,098 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:53,098 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 19:54:53,098 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:53,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:53,098 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-08 19:54:53,098 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:53,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:53,098 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:53,098 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:53,099 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:53,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:53,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:53,099 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:53,099 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:53,099 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:53,199 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:53,200 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:53,204 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:53,205 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:53,206 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:53,206 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 19:54:53,206 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:53,206 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-08 19:54:53,206 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-08 19:54:53,206 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:53,206 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:53,209 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:53,209 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:53,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:53,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:53,209 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:53,209 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:53,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:53,210 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:53,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:53,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:53,210 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:53,210 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-08 19:54:53,210 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:53,210 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-08 19:54:53,210 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:53,210 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-08 19:54:53,211 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:53,211 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:53,211 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:54:53,211 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. started... 2023-03-08 19:54:53,211 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a526a5895c0e84ca75dee0352c5e6b73 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-08 19:54:53,625 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128bba96cb95647424ebb5aa564e82b01b2 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128bba96cb95647424ebb5aa564e82b01b2 2023-03-08 19:54:54,031 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/.tmp/family1/dfc39a33d36f461da09ffdb332786264 2023-03-08 19:54:54,036 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/.tmp/family1/dfc39a33d36f461da09ffdb332786264 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/dfc39a33d36f461da09ffdb332786264 2023-03-08 19:54:54,039 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/dfc39a33d36f461da09ffdb332786264, entries=1, sequenceid=9, filesize=5.0 K 2023-03-08 19:54:54,040 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for a526a5895c0e84ca75dee0352c5e6b73 in 829ms, sequenceid=9, compaction requested=false 2023-03-08 19:54:54,040 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a526a5895c0e84ca75dee0352c5e6b73: 2023-03-08 19:54:54,040 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:54:54,040 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:54,040 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:54,040 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:54,040 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-08 19:54:54,040 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-08 19:54:54,042 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:54,042 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:54,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:54,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:54,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:54,043 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:54,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:54,043 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:54,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:54,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:54,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:54,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:54,044 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-08 19:54:54,044 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3070b8c2[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:54,044 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-08 19:54:54,044 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:54,045 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:54,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:54,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:54,045 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-08 19:54:54,045 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-08 19:54:54,045 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:54,045 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:54,045 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:54,047 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:54,047 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-08 19:54:54,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:54,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:54,047 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:54,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:54,047 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-08 19:54:54,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:54,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:54,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:54,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:54,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:54,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:54,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:54,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:54,049 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-08 19:54:54,049 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-08 19:54:54,049 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:54,049 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:54,049 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:54,049 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:54,052 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:54,052 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:54,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:54,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:54,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:54,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:54,052 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:54,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:54,053 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:54,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:54,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:54,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:54,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:54,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:54,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:54,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:54,054 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:54,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:54,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:54,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:54,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:54,056 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:54,056 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:54,056 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:54,056 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:54,056 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:54,056 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:54,056 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:54,056 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:54,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:54,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:54,056 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 19:54:54,056 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:54,056 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:54,056 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-08 19:54:54,056 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:54,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:54,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:54,057 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:54,057 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:54,157 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:54,158 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:54,162 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:54,163 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:54,163 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:54,163 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 19:54:54,163 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:54,163 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-08 19:54:54,163 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-08 19:54:54,164 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:54,164 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:54,165 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:54,165 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:54,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:54,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:54,166 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:54,166 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:54,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:54,166 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:54,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:54,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:54,166 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:54,166 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-08 19:54:54,167 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:54,167 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-08 19:54:54,167 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:54,167 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-08 19:54:54,167 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:54,167 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:54,167 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:54:54,167 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. started... 2023-03-08 19:54:54,167 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a526a5895c0e84ca75dee0352c5e6b73 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:54:54,301 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 19:54:54,579 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201c99b30cbd90a400dab4d758d6b9e4989 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201c99b30cbd90a400dab4d758d6b9e4989 2023-03-08 19:54:54,985 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/.tmp/family1/f911510646cd436988f0c5269f41c313 2023-03-08 19:54:54,990 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/.tmp/family1/f911510646cd436988f0c5269f41c313 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/f911510646cd436988f0c5269f41c313 2023-03-08 19:54:54,993 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/f911510646cd436988f0c5269f41c313, entries=1, sequenceid=13, filesize=5.0 K 2023-03-08 19:54:54,994 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a526a5895c0e84ca75dee0352c5e6b73 in 827ms, sequenceid=13, compaction requested=true 2023-03-08 19:54:54,994 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a526a5895c0e84ca75dee0352c5e6b73: 2023-03-08 19:54:54,994 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:54:54,994 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:54,994 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:54,994 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:54,994 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-08 19:54:54,994 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-08 19:54:54,996 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:54,996 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:54,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:54,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:54,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:54,996 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:54,996 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:54,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:54,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:54,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:54,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:54,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:54,998 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-08 19:54:54,998 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6b889863[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:54,998 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-08 19:54:54,998 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,000 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,000 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-08 19:54:55,000 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-08 19:54:55,000 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:55,000 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:55,000 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:55,001 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-08 19:54:55,001 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:55,002 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:55,002 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-08 19:54:55,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:55,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:55,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:55,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:55,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:55,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:55,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:55,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:55,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:55,004 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-08 19:54:55,004 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-08 19:54:55,004 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:55,004 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:55,004 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,004 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:55,005 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,005 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,005 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:55,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:55,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:55,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:55,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:55,006 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:55,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:55,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:55,007 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:55,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:55,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:55,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,009 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:55,009 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:55,009 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:55,009 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:55,009 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:55,009 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 19:54:55,009 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:55,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:55,009 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,009 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:55,009 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,009 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,009 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,009 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-08 19:54:55,009 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:55,010 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:55,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:55,010 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:55,110 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:55,111 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:55,114 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:55,116 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:55,116 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,116 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 19:54:55,116 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:55,116 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-08 19:54:55,116 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-08 19:54:55,117 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,117 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,119 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:55,119 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:55,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:55,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:55,119 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:55,119 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:55,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,120 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:55,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,120 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,120 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-08 19:54:55,120 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,120 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-08 19:54:55,120 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:55,120 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-08 19:54:55,121 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:55,121 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:55,121 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:54:55,121 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. started... 2023-03-08 19:54:55,121 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a526a5895c0e84ca75dee0352c5e6b73 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:54:55,534 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205255c94ea5c8f4be0b6f098447786d41b to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205255c94ea5c8f4be0b6f098447786d41b 2023-03-08 19:54:55,939 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/.tmp/family1/ca017011930b466f83ac53a7ce2d7ab4 2023-03-08 19:54:55,944 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/.tmp/family1/ca017011930b466f83ac53a7ce2d7ab4 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/ca017011930b466f83ac53a7ce2d7ab4 2023-03-08 19:54:55,948 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/ca017011930b466f83ac53a7ce2d7ab4, entries=1, sequenceid=17, filesize=5.0 K 2023-03-08 19:54:55,949 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a526a5895c0e84ca75dee0352c5e6b73 in 828ms, sequenceid=17, compaction requested=true 2023-03-08 19:54:55,949 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a526a5895c0e84ca75dee0352c5e6b73: 2023-03-08 19:54:55,949 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:54:55,949 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:55,949 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:55,949 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:55,949 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-08 19:54:55,949 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-08 19:54:55,951 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:55,951 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:55,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:55,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:55,951 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,951 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:55,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:55,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:55,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:55,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:55,952 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-08 19:54:55,952 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@24a0e61[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:55,952 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-08 19:54:55,953 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,957 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,957 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-08 19:54:55,958 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:55,958 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:55,958 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-08 19:54:55,958 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:55,959 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:55,959 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-08 19:54:55,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:55,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:55,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:55,959 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:55,959 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-08 19:54:55,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:55,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:55,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:55,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:55,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:55,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:55,962 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-08 19:54:55,962 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-08 19:54:55,962 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:55,962 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:55,962 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,962 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:55,963 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,963 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:55,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:55,963 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:55,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:55,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:55,964 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:55,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:55,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:55,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,965 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:55,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:55,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:55,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:55,968 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:55,968 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,968 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,968 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:55,968 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:55,968 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:55,968 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 19:54:55,969 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:55,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:55,969 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-08 19:54:55,969 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:55,968 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:55,969 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,969 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:55,969 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:55,969 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:55,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:55,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:56,069 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:56,070 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:56,074 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:56,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:56,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:56,076 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 19:54:56,076 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:56,076 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-08 19:54:56,076 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-08 19:54:56,076 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:56,076 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:56,078 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:56,078 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:56,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:56,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:56,079 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:56,079 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:56,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:56,079 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:56,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:56,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:56,080 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:56,080 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-08 19:54:56,080 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:56,080 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-08 19:54:56,080 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:56,080 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-08 19:54:56,080 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:56,080 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:56,080 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:54:56,080 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. started... 2023-03-08 19:54:56,080 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a526a5895c0e84ca75dee0352c5e6b73 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:54:56,492 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228509d1ff0f93a4a5b9f7c03b1819f26e9 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228509d1ff0f93a4a5b9f7c03b1819f26e9 2023-03-08 19:54:56,898 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/.tmp/family1/b94917f4545f4274824e57dc0db434fa 2023-03-08 19:54:56,903 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/.tmp/family1/b94917f4545f4274824e57dc0db434fa as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/b94917f4545f4274824e57dc0db434fa 2023-03-08 19:54:56,907 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/b94917f4545f4274824e57dc0db434fa, entries=1, sequenceid=21, filesize=5.0 K 2023-03-08 19:54:56,908 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a526a5895c0e84ca75dee0352c5e6b73 in 828ms, sequenceid=21, compaction requested=true 2023-03-08 19:54:56,908 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a526a5895c0e84ca75dee0352c5e6b73: 2023-03-08 19:54:56,908 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:54:56,908 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:56,908 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:56,908 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:56,908 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-08 19:54:56,908 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-08 19:54:56,910 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:56,910 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:56,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:56,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:56,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:56,910 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:56,910 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:56,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:56,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:56,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:56,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:56,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:56,912 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-08 19:54:56,912 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@139b120[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:56,912 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-08 19:54:56,912 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:56,913 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:56,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:56,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:56,913 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-08 19:54:56,913 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-08 19:54:56,913 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:56,913 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:56,913 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:56,920 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:56,920 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-08 19:54:56,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:56,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:56,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:56,920 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:56,920 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-08 19:54:56,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:56,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:56,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:56,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:56,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:56,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:56,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:56,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:56,922 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-08 19:54:56,922 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-08 19:54:56,922 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:56,922 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:56,922 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:56,922 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:56,924 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:56,924 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:56,924 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:56,924 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:56,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:56,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:56,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:56,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:56,925 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:56,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:56,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:56,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:56,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:56,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:56,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:56,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:56,926 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:56,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:56,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:56,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:56,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:56,928 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:56,928 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:56,928 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:56,928 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:56,928 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 19:54:56,928 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:56,928 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-08 19:54:56,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:56,928 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:56,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:56,928 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:56,928 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:56,928 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:56,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:56,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:56,928 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:56,928 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:56,929 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:56,929 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,029 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:57,030 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:57,033 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:57,035 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:57,035 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,035 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 19:54:57,035 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:57,035 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-08 19:54:57,035 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-08 19:54:57,035 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,036 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,037 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:57,037 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:57,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:57,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:57,037 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:57,037 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:57,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,038 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:57,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,038 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,038 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-08 19:54:57,038 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,038 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-08 19:54:57,038 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:57,038 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-08 19:54:57,039 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:57,039 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:57,039 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:54:57,039 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. started... 2023-03-08 19:54:57,039 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a526a5895c0e84ca75dee0352c5e6b73 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:54:57,451 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123102d48a0d6df04f649b446a9bae9460be to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015123102d48a0d6df04f649b446a9bae9460be 2023-03-08 19:54:57,856 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/.tmp/family1/b9b9708f95934a24a3c7d8933d1cf277 2023-03-08 19:54:57,861 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/.tmp/family1/b9b9708f95934a24a3c7d8933d1cf277 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/b9b9708f95934a24a3c7d8933d1cf277 2023-03-08 19:54:57,866 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/b9b9708f95934a24a3c7d8933d1cf277, entries=1, sequenceid=25, filesize=5.0 K 2023-03-08 19:54:57,866 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a526a5895c0e84ca75dee0352c5e6b73 in 827ms, sequenceid=25, compaction requested=true 2023-03-08 19:54:57,866 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a526a5895c0e84ca75dee0352c5e6b73: 2023-03-08 19:54:57,867 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:54:57,867 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:57,867 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:57,867 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:57,867 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-08 19:54:57,867 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-08 19:54:57,868 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,868 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:57,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:57,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:57,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:57,869 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,869 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:57,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:57,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:57,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:57,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:57,870 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-08 19:54:57,870 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@343abf6d[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:57,870 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-08 19:54:57,870 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,871 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,872 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-08 19:54:57,872 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:57,872 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:57,872 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-08 19:54:57,872 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:57,873 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:57,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:57,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:57,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:57,873 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-08 19:54:57,873 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:57,873 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-08 19:54:57,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:57,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:57,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:57,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:57,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:57,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:57,875 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-08 19:54:57,875 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-08 19:54:57,875 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:57,875 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:57,875 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,875 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:57,878 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,878 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:57,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:57,878 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:57,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,879 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:57,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:57,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:57,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:57,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:57,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,880 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:57,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:57,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:57,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:57,885 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:57,885 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:57,885 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:57,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:57,885 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:57,885 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 19:54:57,885 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:57,885 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:57,885 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,885 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:57,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:57,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:57,885 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,885 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,885 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:57,885 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:57,885 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,986 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:57,987 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:57,990 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:57,991 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:57,991 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,992 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 19:54:57,992 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:57,992 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-08 19:54:57,992 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-08 19:54:57,992 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,992 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,996 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:57,996 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:57,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:57,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:57,996 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:57,996 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:57,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,997 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:57,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,997 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,997 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-08 19:54:57,997 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:57,997 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-08 19:54:57,997 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:57,997 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-08 19:54:57,998 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:57,998 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:57,998 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:54:57,998 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. started... 2023-03-08 19:54:57,998 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a526a5895c0e84ca75dee0352c5e6b73 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:54:58,410 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010149cfbea3c42c4a3a9da07ae09ba551af to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010149cfbea3c42c4a3a9da07ae09ba551af 2023-03-08 19:54:58,816 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/.tmp/family1/f3414a64a05a4be2a2ff6b37b0d344e3 2023-03-08 19:54:58,823 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/.tmp/family1/f3414a64a05a4be2a2ff6b37b0d344e3 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/f3414a64a05a4be2a2ff6b37b0d344e3 2023-03-08 19:54:58,827 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/f3414a64a05a4be2a2ff6b37b0d344e3, entries=1, sequenceid=29, filesize=5.0 K 2023-03-08 19:54:58,828 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a526a5895c0e84ca75dee0352c5e6b73 in 830ms, sequenceid=29, compaction requested=true 2023-03-08 19:54:58,828 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a526a5895c0e84ca75dee0352c5e6b73: 2023-03-08 19:54:58,828 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:54:58,828 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:58,828 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:58,828 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:58,828 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-08 19:54:58,828 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-08 19:54:58,830 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:58,830 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:58,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:58,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:58,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:58,830 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:58,830 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:58,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:58,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:58,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:58,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:58,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:58,831 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-08 19:54:58,831 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1bde7209[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:58,831 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-08 19:54:58,832 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:58,833 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:58,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:58,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:58,833 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-08 19:54:58,833 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-08 19:54:58,833 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:58,833 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:58,833 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:58,835 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:58,835 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-08 19:54:58,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:58,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:58,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:58,835 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:58,836 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-08 19:54:58,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:58,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:58,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:58,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:58,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:58,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:58,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:58,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:58,838 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-08 19:54:58,838 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-08 19:54:58,838 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:58,838 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:58,838 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:58,838 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:58,839 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:58,839 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:58,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:58,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:58,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:58,839 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:58,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:58,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:58,840 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:58,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:58,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:58,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:58,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:58,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:58,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:58,841 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:58,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:58,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:58,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:58,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:58,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:58,843 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:58,843 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:58,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:58,843 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:58,843 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:58,843 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:58,843 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 19:54:58,843 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:58,843 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:58,843 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:58,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:58,844 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-08 19:54:58,844 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:58,844 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:58,844 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:58,844 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:58,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:58,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:58,844 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:58,944 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:58,945 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:58,952 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:58,961 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:58,961 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:58,961 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 19:54:58,961 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:58,961 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-08 19:54:58,961 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-08 19:54:58,962 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:58,962 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:58,964 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:58,964 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:58,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:58,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:58,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:58,964 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:58,964 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:58,964 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:58,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:58,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:58,965 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:58,968 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-08 19:54:58,968 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:58,968 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-08 19:54:58,968 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:58,968 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-08 19:54:58,968 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:58,968 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:58,968 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:54:58,968 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. started... 2023-03-08 19:54:58,968 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a526a5895c0e84ca75dee0352c5e6b73 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:54:59,385 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601034913ccda6447450097da80aeb58c8a68 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601034913ccda6447450097da80aeb58c8a68 2023-03-08 19:54:59,791 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/.tmp/family1/28585fcb78d54590bd2d48504b53f913 2023-03-08 19:54:59,796 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/.tmp/family1/28585fcb78d54590bd2d48504b53f913 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/28585fcb78d54590bd2d48504b53f913 2023-03-08 19:54:59,800 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/28585fcb78d54590bd2d48504b53f913, entries=1, sequenceid=33, filesize=5.0 K 2023-03-08 19:54:59,801 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a526a5895c0e84ca75dee0352c5e6b73 in 833ms, sequenceid=33, compaction requested=true 2023-03-08 19:54:59,801 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a526a5895c0e84ca75dee0352c5e6b73: 2023-03-08 19:54:59,801 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:54:59,801 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:54:59,801 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:54:59,801 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:59,801 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-08 19:54:59,801 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-08 19:54:59,803 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:59,803 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:59,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:59,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:59,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:59,804 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:59,804 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:54:59,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:59,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:59,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:59,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:59,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:59,805 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-08 19:54:59,805 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@49b55e15[Count = 0] remaining members to acquire global barrier 2023-03-08 19:54:59,805 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-08 19:54:59,805 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:59,806 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:59,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:59,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:59,806 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-08 19:54:59,806 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-08 19:54:59,806 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:54:59,806 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:59,806 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:54:59,808 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-08 19:54:59,808 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:59,808 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:59,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:59,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:59,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:59,808 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-08 19:54:59,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:59,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:59,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:59,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:59,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:59,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:59,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:59,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:54:59,810 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-08 19:54:59,810 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-08 19:54:59,810 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:54:59,810 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:54:59,810 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:59,810 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:54:59,812 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:59,812 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:59,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:59,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:59,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:54:59,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:54:59,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:59,812 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:59,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:54:59,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:59,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:59,812 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:59,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:59,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:59,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:54:59,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:59,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:59,813 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:59,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:54:59,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:59,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:59,816 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:59,816 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:59,816 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:59,816 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:59,816 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 19:54:59,816 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:54:59,816 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:54:59,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:59,816 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:54:59,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:59,816 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-08 19:54:59,816 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:59,817 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-08 19:54:59,817 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:59,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:54:59,817 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:59,817 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:54:59,817 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:59,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:54:59,917 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:54:59,918 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:54:59,922 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:54:59,923 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-08 19:54:59,923 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:59,923 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 19:54:59,923 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:59,923 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-08 19:54:59,923 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-08 19:54:59,924 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:59,924 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:59,925 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:59,925 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:54:59,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:54:59,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:54:59,925 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:54:59,925 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:54:59,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:59,926 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:59,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:54:59,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:59,926 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:59,926 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-08 19:54:59,926 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:54:59,927 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-08 19:54:59,927 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:54:59,927 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-08 19:54:59,927 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:54:59,927 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:54:59,927 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:54:59,927 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. started... 2023-03-08 19:54:59,927 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a526a5895c0e84ca75dee0352c5e6b73 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:55:00,344 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509075554f4b955e14502ba78a927055f8f30 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201509075554f4b955e14502ba78a927055f8f30 2023-03-08 19:55:00,366 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x511c347f to 127.0.0.1:56751 2023-03-08 19:55:00,367 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 19:55:00,367 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 19:55:00,367 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6ea2c454 to 127.0.0.1:56751 2023-03-08 19:55:00,367 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 19:55:00,367 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 19:55:00,368 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0279552a to 127.0.0.1:56751 2023-03-08 19:55:00,368 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 19:55:00,368 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 19:55:00,368 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x646f78e0 to 127.0.0.1:56751 2023-03-08 19:55:00,368 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 19:55:00,771 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/.tmp/family1/f36160d9be284db9a6ca48ac0716858a 2023-03-08 19:55:00,776 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/.tmp/family1/f36160d9be284db9a6ca48ac0716858a as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/f36160d9be284db9a6ca48ac0716858a 2023-03-08 19:55:00,780 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/f36160d9be284db9a6ca48ac0716858a, entries=1, sequenceid=37, filesize=5.0 K 2023-03-08 19:55:00,781 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a526a5895c0e84ca75dee0352c5e6b73 in 854ms, sequenceid=37, compaction requested=true 2023-03-08 19:55:00,781 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a526a5895c0e84ca75dee0352c5e6b73: 2023-03-08 19:55:00,781 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:55:00,781 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:55:00,781 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:55:00,781 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:00,782 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-08 19:55:00,782 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-08 19:55:00,783 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:00,784 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:00,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:00,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:00,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:00,784 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:00,784 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:55:00,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:00,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:00,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:00,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:00,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:00,785 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-08 19:55:00,785 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@54946b2[Count = 0] remaining members to acquire global barrier 2023-03-08 19:55:00,785 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-08 19:55:00,786 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:00,787 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:00,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:00,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:00,787 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:00,787 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-08 19:55:00,787 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:55:00,787 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-08 19:55:00,787 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:55:00,789 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-08 19:55:00,789 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:55:00,789 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:00,789 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-08 19:55:00,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:00,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:00,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:00,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:00,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:00,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:00,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:00,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:00,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:00,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:00,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:55:00,792 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-08 19:55:00,792 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-08 19:55:00,792 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:55:00,792 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:55:00,792 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:00,792 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:55:00,794 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:00,794 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:00,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:00,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:00,794 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:55:00,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:00,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:00,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:00,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:55:00,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:55:00,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:00,794 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:00,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:00,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:00,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:00,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:00,796 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:00,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:00,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:00,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:00,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:00,798 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:00,798 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:00,798 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:00,799 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:00,799 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:55:00,798 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:55:00,799 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-08 19:55:00,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:55:00,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:55:00,799 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:55:00,799 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 19:55:00,799 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:55:00,799 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:00,799 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:00,799 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:00,799 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-08 19:55:00,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:55:00,799 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:55:00,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:55:00,900 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:55:00,901 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:55:00,904 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:55:00,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-08 19:55:00,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:00,906 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 19:55:00,906 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:55:00,906 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-08 19:55:00,906 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-08 19:55:00,907 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:00,907 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:00,908 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:55:00,908 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:00,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:55:00,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:55:00,908 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:00,908 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:55:00,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:00,909 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:00,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:55:00,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:00,909 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:00,910 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-08 19:55:00,910 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:00,910 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-08 19:55:00,910 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:55:00,910 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-08 19:55:00,910 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:55:00,910 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:55:00,910 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:55:00,910 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. started... 2023-03-08 19:55:00,910 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a526a5895c0e84ca75dee0352c5e6b73 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:55:01,322 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120376678335cf44ace9a51f6f0b243379a to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120376678335cf44ace9a51f6f0b243379a 2023-03-08 19:55:01,728 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/.tmp/family1/380b50c19f3a4cd2b60fcdddc583dbce 2023-03-08 19:55:01,732 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/.tmp/family1/380b50c19f3a4cd2b60fcdddc583dbce as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/380b50c19f3a4cd2b60fcdddc583dbce 2023-03-08 19:55:01,736 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/380b50c19f3a4cd2b60fcdddc583dbce, entries=1, sequenceid=41, filesize=5.0 K 2023-03-08 19:55:01,736 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a526a5895c0e84ca75dee0352c5e6b73 in 826ms, sequenceid=41, compaction requested=true 2023-03-08 19:55:01,737 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a526a5895c0e84ca75dee0352c5e6b73: 2023-03-08 19:55:01,737 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:55:01,737 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:55:01,737 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:55:01,737 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:01,737 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-08 19:55:01,737 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-08 19:55:01,739 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:01,739 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:01,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:01,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:01,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:01,740 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:01,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:01,740 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:55:01,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:01,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:01,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:01,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:01,741 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-08 19:55:01,741 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@646940a1[Count = 0] remaining members to acquire global barrier 2023-03-08 19:55:01,741 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-08 19:55:01,741 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:01,742 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:01,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:01,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:01,742 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-08 19:55:01,742 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-08 19:55:01,742 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:55:01,743 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:01,743 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:55:01,744 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:01,744 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-08 19:55:01,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:01,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:01,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:01,744 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:55:01,744 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-08 19:55:01,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:01,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:01,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:01,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:01,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:01,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:01,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:01,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:55:01,746 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-08 19:55:01,746 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-08 19:55:01,746 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:55:01,746 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:55:01,746 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:01,746 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:55:01,747 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:01,748 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:01,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:01,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:01,748 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:01,748 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:55:01,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:01,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:01,748 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:55:01,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:55:01,748 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:01,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:01,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:01,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:01,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:01,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:01,749 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:01,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:01,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:01,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:01,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:01,753 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:55:01,753 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:01,753 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:55:01,753 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:01,753 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:01,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:55:01,753 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-08 19:55:01,753 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:55:01,753 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-08 19:55:01,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:55:01,753 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:55:01,753 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:01,753 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:01,753 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:01,753 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-08 19:55:01,753 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:01,753 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:55:01,754 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:55:01,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:55:01,854 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:55:01,855 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:55:01,856 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39989] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:01,858 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] procedure2.ProcedureExecutor(1029): Stored pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-08 19:55:01,858 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-08 19:55:01,859 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 19:55:01,859 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=58, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678305901859 2023-03-08 19:55:01,862 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 19:55:01,862 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-03-08 19:55:01,863 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 19:55:01,863 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] zookeeper.ReadOnlyZKClient(139): Connect 0x76bcf796 to 127.0.0.1:56751 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 19:55:01,867 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6e65a873, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 19:55:01,868 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 19:55:02,706 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103f80a5b9447514d1bbce1991726395913 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103f80a5b9447514d1bbce1991726395913 2023-03-08 19:55:02,707 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 19:55:02,721 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 19:55:02,722 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33056, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 19:55:02,731 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 19:55:02,732 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55296, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 19:55:02,742 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/ea67eba947f14c7383a72a2809a68512 first=Optional[r3] last=Optional[r6] 2023-03-08 19:55:02,743 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73., hostname=jenkins-hbase4.apache.org,33367,1678305218680, seqNum=2 for row with hfile group [{family1,hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/ea67eba947f14c7383a72a2809a68512}] 2023-03-08 19:55:02,747 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(763): Validating hfile at hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/ea67eba947f14c7383a72a2809a68512 for inclusion in a526a5895c0e84ca75dee0352c5e6b73/family1 2023-03-08 19:55:02,749 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-08 19:55:02,749 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 19:55:02,749 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(2446): Flush status journal for a526a5895c0e84ca75dee0352c5e6b73: 2023-03-08 19:55:02,750 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/ea67eba947f14c7383a72a2809a68512 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ldpplktpcvo5gse8c75kvt7nlcu92l68mkbor9i93g7ng69e2f5vshbo6qlptb48/family1/ea67eba947f14c7383a72a2809a68512 2023-03-08 19:55:02,752 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ldpplktpcvo5gse8c75kvt7nlcu92l68mkbor9i93g7ng69e2f5vshbo6qlptb48/family1/ea67eba947f14c7383a72a2809a68512 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/c470239dc4ca4caf80d6f7a46ddd87fc_SeqId_44_ 2023-03-08 19:55:02,753 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(849): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ldpplktpcvo5gse8c75kvt7nlcu92l68mkbor9i93g7ng69e2f5vshbo6qlptb48/family1/ea67eba947f14c7383a72a2809a68512 into a526a5895c0e84ca75dee0352c5e6b73/family1 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/c470239dc4ca4caf80d6f7a46ddd87fc_SeqId_44_ - updating store file list. 2023-03-08 19:55:02,757 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(882): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/c470239dc4ca4caf80d6f7a46ddd87fc_SeqId_44_ into a526a5895c0e84ca75dee0352c5e6b73/family1 2023-03-08 19:55:02,757 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(855): Successfully loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ldpplktpcvo5gse8c75kvt7nlcu92l68mkbor9i93g7ng69e2f5vshbo6qlptb48/family1/ea67eba947f14c7383a72a2809a68512 into a526a5895c0e84ca75dee0352c5e6b73/family1 (new location: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/c470239dc4ca4caf80d6f7a46ddd87fc_SeqId_44_) 2023-03-08 19:55:02,757 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ldpplktpcvo5gse8c75kvt7nlcu92l68mkbor9i93g7ng69e2f5vshbo6qlptb48/family1/ea67eba947f14c7383a72a2809a68512 2023-03-08 19:55:02,757 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-08 19:55:02,760 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51552 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-08 19:55:02,760 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(1912): a526a5895c0e84ca75dee0352c5e6b73/family1 is initiating minor compaction 2023-03-08 19:55:02,760 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@7905a5d0; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 19:55:02,760 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of a526a5895c0e84ca75dee0352c5e6b73/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:55:02,760 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/dfc39a33d36f461da09ffdb332786264, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/f911510646cd436988f0c5269f41c313, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/ca017011930b466f83ac53a7ce2d7ab4, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/b94917f4545f4274824e57dc0db434fa, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/b9b9708f95934a24a3c7d8933d1cf277, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/f3414a64a05a4be2a2ff6b37b0d344e3, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/28585fcb78d54590bd2d48504b53f913, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/f36160d9be284db9a6ca48ac0716858a, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/380b50c19f3a4cd2b60fcdddc583dbce, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/c470239dc4ca4caf80d6f7a46ddd87fc_SeqId_44_] into tmpdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/.tmp, totalSize=50.3 K 2023-03-08 19:55:02,761 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting dfc39a33d36f461da09ffdb332786264, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-08 19:55:02,761 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting f911510646cd436988f0c5269f41c313, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-08 19:55:02,761 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting ca017011930b466f83ac53a7ce2d7ab4, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-08 19:55:02,761 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting b94917f4545f4274824e57dc0db434fa, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-08 19:55:02,761 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting b9b9708f95934a24a3c7d8933d1cf277, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-08 19:55:02,761 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting f3414a64a05a4be2a2ff6b37b0d344e3, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-08 19:55:02,761 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 28585fcb78d54590bd2d48504b53f913, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-08 19:55:02,761 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting f36160d9be284db9a6ca48ac0716858a, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-08 19:55:02,761 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 380b50c19f3a4cd2b60fcdddc583dbce, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-08 19:55:02,761 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting c470239dc4ca4caf80d6f7a46ddd87fc_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-08 19:55:02,762 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 19:55:02,763 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:55:02,764 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228509d1ff0f93a4a5b9f7c03b1819f26e9 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228509d1ff0f93a4a5b9f7c03b1819f26e9 2023-03-08 19:55:02,765 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015123102d48a0d6df04f649b446a9bae9460be to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015123102d48a0d6df04f649b446a9bae9460be 2023-03-08 19:55:02,767 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010149cfbea3c42c4a3a9da07ae09ba551af to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010149cfbea3c42c4a3a9da07ae09ba551af 2023-03-08 19:55:02,768 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601034913ccda6447450097da80aeb58c8a68 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601034913ccda6447450097da80aeb58c8a68 2023-03-08 19:55:02,768 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 4 expired mob files are deleted 2023-03-08 19:55:02,768 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-08 19:55:02,769 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 19:55:02,769 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 19:55:02,769 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 19:55:02,769 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 19:55:02,769 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 19:55:02,769 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 19:55:02,769 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 19:55:02,769 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 19:55:02,806 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] throttle.PressureAwareThroughputController(145): a526a5895c0e84ca75dee0352c5e6b73#family1#compaction#73 average throughput is 0.79 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 19:55:03,235 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512051fbac2d2e2e84de584cca20fe988d022 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512051fbac2d2e2e84de584cca20fe988d022 2023-03-08 19:55:03,236 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 19:55:03,278 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/b39ab81fb8324120b9b826ca334b3cca first=Optional[r00] last=Optional[r2] 2023-03-08 19:55:03,279 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73., hostname=jenkins-hbase4.apache.org,33367,1678305218680, seqNum=2 for row with hfile group [{family1,hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/b39ab81fb8324120b9b826ca334b3cca}] 2023-03-08 19:55:03,282 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(763): Validating hfile at hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/b39ab81fb8324120b9b826ca334b3cca for inclusion in a526a5895c0e84ca75dee0352c5e6b73/family1 2023-03-08 19:55:03,285 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-08 19:55:03,285 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 19:55:03,285 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(2446): Flush status journal for a526a5895c0e84ca75dee0352c5e6b73: 2023-03-08 19:55:03,287 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/b39ab81fb8324120b9b826ca334b3cca to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__2cg3g0bnp1e8njec3d39orf6q3kbu5oifoq3l3lprgn97nd2me4gpd9bt2vk4jc7/family1/b39ab81fb8324120b9b826ca334b3cca 2023-03-08 19:55:03,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__2cg3g0bnp1e8njec3d39orf6q3kbu5oifoq3l3lprgn97nd2me4gpd9bt2vk4jc7/family1/b39ab81fb8324120b9b826ca334b3cca as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/f09f89d84d194febbdd35590914ddd92_SeqId_46_ 2023-03-08 19:55:03,290 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(849): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__2cg3g0bnp1e8njec3d39orf6q3kbu5oifoq3l3lprgn97nd2me4gpd9bt2vk4jc7/family1/b39ab81fb8324120b9b826ca334b3cca into a526a5895c0e84ca75dee0352c5e6b73/family1 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/f09f89d84d194febbdd35590914ddd92_SeqId_46_ - updating store file list. 2023-03-08 19:55:03,294 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(882): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/f09f89d84d194febbdd35590914ddd92_SeqId_46_ into a526a5895c0e84ca75dee0352c5e6b73/family1 2023-03-08 19:55:03,294 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(855): Successfully loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__2cg3g0bnp1e8njec3d39orf6q3kbu5oifoq3l3lprgn97nd2me4gpd9bt2vk4jc7/family1/b39ab81fb8324120b9b826ca334b3cca into a526a5895c0e84ca75dee0352c5e6b73/family1 (new location: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/f09f89d84d194febbdd35590914ddd92_SeqId_46_) 2023-03-08 19:55:03,295 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__2cg3g0bnp1e8njec3d39orf6q3kbu5oifoq3l3lprgn97nd2me4gpd9bt2vk4jc7/family1/b39ab81fb8324120b9b826ca334b3cca 2023-03-08 19:55:03,297 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 19:55:03,298 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:55:03,299 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015113037786975b822441483b3eea546c3754d to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015113037786975b822441483b3eea546c3754d 2023-03-08 19:55:03,301 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201c99b30cbd90a400dab4d758d6b9e4989 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201c99b30cbd90a400dab4d758d6b9e4989 2023-03-08 19:55:03,302 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205255c94ea5c8f4be0b6f098447786d41b to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205255c94ea5c8f4be0b6f098447786d41b 2023-03-08 19:55:03,302 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 3 expired mob files are deleted 2023-03-08 19:55:03,302 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-08 19:55:03,302 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 19:55:03,303 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 19:55:03,303 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 19:55:03,304 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=58, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-08 19:55:03,305 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 1.4470 sec 2023-03-08 19:55:03,385 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-08 19:55:03,386 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure2.ProcedureExecutor(1029): Stored pid=59, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:03,629 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/.tmp/family1/feb0948d958440778bf46f36b4590ee7 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/feb0948d958440778bf46f36b4590ee7 2023-03-08 19:55:03,633 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in a526a5895c0e84ca75dee0352c5e6b73/family1 of a526a5895c0e84ca75dee0352c5e6b73 into feb0948d958440778bf46f36b4590ee7(size=6.8 K), total size for store is 388.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 19:55:03,633 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for a526a5895c0e84ca75dee0352c5e6b73: 2023-03-08 19:55:03,633 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73., storeName=family1, fileCount=10, fileSize=50.3 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1678305302760; duration=0sec 2023-03-08 19:55:03,633 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 19:55:03,725 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/dfc39a33d36f461da09ffdb332786264, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/f911510646cd436988f0c5269f41c313, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/ca017011930b466f83ac53a7ce2d7ab4, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/b94917f4545f4274824e57dc0db434fa, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/b9b9708f95934a24a3c7d8933d1cf277, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/f3414a64a05a4be2a2ff6b37b0d344e3, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/28585fcb78d54590bd2d48504b53f913, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/f36160d9be284db9a6ca48ac0716858a, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/380b50c19f3a4cd2b60fcdddc583dbce, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/c470239dc4ca4caf80d6f7a46ddd87fc_SeqId_44_] to archive 2023-03-08 19:55:03,726 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:55:03,727 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/dfc39a33d36f461da09ffdb332786264 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/dfc39a33d36f461da09ffdb332786264 2023-03-08 19:55:03,729 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/f911510646cd436988f0c5269f41c313 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/f911510646cd436988f0c5269f41c313 2023-03-08 19:55:03,730 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/ca017011930b466f83ac53a7ce2d7ab4 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/ca017011930b466f83ac53a7ce2d7ab4 2023-03-08 19:55:03,731 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/b94917f4545f4274824e57dc0db434fa to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/b94917f4545f4274824e57dc0db434fa 2023-03-08 19:55:03,732 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/b9b9708f95934a24a3c7d8933d1cf277 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/b9b9708f95934a24a3c7d8933d1cf277 2023-03-08 19:55:03,732 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/f3414a64a05a4be2a2ff6b37b0d344e3 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/f3414a64a05a4be2a2ff6b37b0d344e3 2023-03-08 19:55:03,733 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/28585fcb78d54590bd2d48504b53f913 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/28585fcb78d54590bd2d48504b53f913 2023-03-08 19:55:03,734 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/f36160d9be284db9a6ca48ac0716858a to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/f36160d9be284db9a6ca48ac0716858a 2023-03-08 19:55:03,735 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/380b50c19f3a4cd2b60fcdddc583dbce to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/380b50c19f3a4cd2b60fcdddc583dbce 2023-03-08 19:55:03,736 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/c470239dc4ca4caf80d6f7a46ddd87fc_SeqId_44_ to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/c470239dc4ca4caf80d6f7a46ddd87fc_SeqId_44_ 2023-03-08 19:55:03,795 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-03-08 19:55:03,796 DEBUG [PEWorker-4] util.FSTableDescriptors(519): Deleted hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-03-08 19:55:03,796 INFO [PEWorker-4] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-03-08 19:55:03,798 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=60, ppid=59, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-03-08 19:55:03,800 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=a526a5895c0e84ca75dee0352c5e6b73, REOPEN/MOVE}] 2023-03-08 19:55:03,801 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=a526a5895c0e84ca75dee0352c5e6b73, REOPEN/MOVE 2023-03-08 19:55:03,802 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=a526a5895c0e84ca75dee0352c5e6b73, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:03,802 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678305303802"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678305303802"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678305303802"}]},"ts":"1678305303802"} 2023-03-08 19:55:03,803 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=61, state=RUNNABLE; CloseRegionProcedure a526a5895c0e84ca75dee0352c5e6b73, server=jenkins-hbase4.apache.org,33367,1678305218680}] 2023-03-08 19:55:03,955 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close a526a5895c0e84ca75dee0352c5e6b73 2023-03-08 19:55:03,955 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a526a5895c0e84ca75dee0352c5e6b73, disabling compactions & flushes 2023-03-08 19:55:03,955 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:55:03,955 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:55:03,955 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. after waiting 0 ms 2023-03-08 19:55:03,955 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:55:03,962 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/recovered.edits/49.seqid, newMaxSeqId=49, maxSeqId=1 2023-03-08 19:55:03,963 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:55:03,963 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a526a5895c0e84ca75dee0352c5e6b73: 2023-03-08 19:55:03,963 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: a526a5895c0e84ca75dee0352c5e6b73 to self. 2023-03-08 19:55:03,964 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed a526a5895c0e84ca75dee0352c5e6b73 2023-03-08 19:55:03,964 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=a526a5895c0e84ca75dee0352c5e6b73, regionState=CLOSED 2023-03-08 19:55:03,965 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678305303964"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678305303964"}]},"ts":"1678305303964"} 2023-03-08 19:55:03,967 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=61 2023-03-08 19:55:03,967 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=61, state=SUCCESS; CloseRegionProcedure a526a5895c0e84ca75dee0352c5e6b73, server=jenkins-hbase4.apache.org,33367,1678305218680 in 163 msec 2023-03-08 19:55:03,967 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=a526a5895c0e84ca75dee0352c5e6b73, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,33367,1678305218680; forceNewPlan=false, retain=true 2023-03-08 19:55:04,118 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=a526a5895c0e84ca75dee0352c5e6b73, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:04,118 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678305304118"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678305304118"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678305304118"}]},"ts":"1678305304118"} 2023-03-08 19:55:04,120 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=63, ppid=61, state=RUNNABLE; OpenRegionProcedure a526a5895c0e84ca75dee0352c5e6b73, server=jenkins-hbase4.apache.org,33367,1678305218680}] 2023-03-08 19:55:04,274 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:55:04,274 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a526a5895c0e84ca75dee0352c5e6b73, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73.', STARTKEY => '', ENDKEY => ''} 2023-03-08 19:55:04,274 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily a526a5895c0e84ca75dee0352c5e6b73 2023-03-08 19:55:04,274 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:55:04,274 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a526a5895c0e84ca75dee0352c5e6b73 2023-03-08 19:55:04,274 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a526a5895c0e84ca75dee0352c5e6b73 2023-03-08 19:55:04,275 INFO [StoreOpener-a526a5895c0e84ca75dee0352c5e6b73-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a526a5895c0e84ca75dee0352c5e6b73 2023-03-08 19:55:04,276 DEBUG [StoreOpener-a526a5895c0e84ca75dee0352c5e6b73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1 2023-03-08 19:55:04,276 DEBUG [StoreOpener-a526a5895c0e84ca75dee0352c5e6b73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1 2023-03-08 19:55:04,277 INFO [StoreOpener-a526a5895c0e84ca75dee0352c5e6b73-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a526a5895c0e84ca75dee0352c5e6b73 columnFamilyName family1 2023-03-08 19:55:04,281 DEBUG [StoreOpener-a526a5895c0e84ca75dee0352c5e6b73-1] regionserver.HStore(539): loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/ae6f273aeef4415585d3c817d2e83672 2023-03-08 19:55:04,285 DEBUG [StoreOpener-a526a5895c0e84ca75dee0352c5e6b73-1] regionserver.HStore(539): loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/f09f89d84d194febbdd35590914ddd92_SeqId_46_ 2023-03-08 19:55:04,288 DEBUG [StoreOpener-a526a5895c0e84ca75dee0352c5e6b73-1] regionserver.HStore(539): loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/feb0948d958440778bf46f36b4590ee7 2023-03-08 19:55:04,288 INFO [StoreOpener-a526a5895c0e84ca75dee0352c5e6b73-1] regionserver.HStore(310): Store=a526a5895c0e84ca75dee0352c5e6b73/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:55:04,289 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73 2023-03-08 19:55:04,290 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73 2023-03-08 19:55:04,292 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a526a5895c0e84ca75dee0352c5e6b73 2023-03-08 19:55:04,293 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a526a5895c0e84ca75dee0352c5e6b73; next sequenceid=50; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68609314, jitterRate=0.022358447313308716}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 19:55:04,293 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a526a5895c0e84ca75dee0352c5e6b73: 2023-03-08 19:55:04,293 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73., pid=63, masterSystemTime=1678305304271 2023-03-08 19:55:04,293 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 19:55:04,294 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-08 19:55:04,295 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 397422 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-08 19:55:04,295 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:55:04,295 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1912): a526a5895c0e84ca75dee0352c5e6b73/family1 is initiating minor compaction (all files) 2023-03-08 19:55:04,295 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:55:04,296 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of a526a5895c0e84ca75dee0352c5e6b73/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:55:04,296 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/ae6f273aeef4415585d3c817d2e83672, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/feb0948d958440778bf46f36b4590ee7, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/f09f89d84d194febbdd35590914ddd92_SeqId_46_] into tmpdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/.tmp, totalSize=388.1 K 2023-03-08 19:55:04,296 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=a526a5895c0e84ca75dee0352c5e6b73, regionState=OPEN, openSeqNum=50, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:04,296 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting ae6f273aeef4415585d3c817d2e83672, keycount=1000, bloomtype=ROW, size=190.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878843488 2023-03-08 19:55:04,296 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678305304296"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678305304296"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678305304296"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678305304296"}]},"ts":"1678305304296"} 2023-03-08 19:55:04,296 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting feb0948d958440778bf46f36b4590ee7, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621243488 2023-03-08 19:55:04,297 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting f09f89d84d194febbdd35590914ddd92_SeqId_46_, keycount=1002, bloomtype=ROW, size=190.9 K, encoding=NONE, compression=NONE, seqNum=47, earliestPutTs=1448878843488 2023-03-08 19:55:04,299 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=63, resume processing ppid=61 2023-03-08 19:55:04,299 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=63, ppid=61, state=SUCCESS; OpenRegionProcedure a526a5895c0e84ca75dee0352c5e6b73, server=jenkins-hbase4.apache.org,33367,1678305218680 in 178 msec 2023-03-08 19:55:04,301 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=60 2023-03-08 19:55:04,301 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=60, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=a526a5895c0e84ca75dee0352c5e6b73, REOPEN/MOVE in 499 msec 2023-03-08 19:55:04,303 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=60, resume processing ppid=59 2023-03-08 19:55:04,303 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=59, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 503 msec 2023-03-08 19:55:04,305 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=59, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 918 msec 2023-03-08 19:55:04,306 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=59 2023-03-08 19:55:04,307 INFO [Listener at localhost/34049] compactions.TestMobCompactor(278): alter status finished 2023-03-08 19:55:04,308 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39989] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:04,309 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] procedure2.ProcedureExecutor(1029): Stored pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-08 19:55:04,310 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-08 19:55:04,311 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 19:55:04,311 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=64, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678305904311 2023-03-08 19:55:04,312 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 19:55:04,312 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-03-08 19:55:04,312 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] throttle.PressureAwareThroughputController(145): a526a5895c0e84ca75dee0352c5e6b73#family1#compaction#74 average throughput is 22.45 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 19:55:04,312 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 19:55:04,313 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] zookeeper.ReadOnlyZKClient(139): Connect 0x29a03c3d to 127.0.0.1:56751 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 19:55:04,321 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1d5f6d2e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 19:55:04,322 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 19:55:04,322 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 19:55:04,322 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 19:55:04,322 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 19:55:04,322 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 19:55:05,140 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/.tmp/family1/78dd1a89e85d4298a7abf8d79696fc24 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/78dd1a89e85d4298a7abf8d79696fc24 2023-03-08 19:55:05,145 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in a526a5895c0e84ca75dee0352c5e6b73/family1 of a526a5895c0e84ca75dee0352c5e6b73 into 78dd1a89e85d4298a7abf8d79696fc24(size=192.0 K), total size for store is 192.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 19:55:05,145 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for a526a5895c0e84ca75dee0352c5e6b73: 2023-03-08 19:55:05,145 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73., storeName=a526a5895c0e84ca75dee0352c5e6b73/family1, priority=13, startTime=1678305304293; duration=0sec 2023-03-08 19:55:05,145 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 19:55:05,149 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112825d65cd122584c89bac128adfe7b532b to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015112825d65cd122584c89bac128adfe7b532b 2023-03-08 19:55:05,150 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 19:55:05,169 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 19:55:05,170 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33102, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 19:55:05,177 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 19:55:05,178 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43456, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 19:55:05,186 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/e123da30ce2f41428dd443eea0aed6a8 first=Optional[r05] last=Optional[r8] 2023-03-08 19:55:05,187 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73., hostname=jenkins-hbase4.apache.org,33367,1678305218680, seqNum=50 for row with hfile group [{family1,hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/e123da30ce2f41428dd443eea0aed6a8}] 2023-03-08 19:55:05,191 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(763): Validating hfile at hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/e123da30ce2f41428dd443eea0aed6a8 for inclusion in a526a5895c0e84ca75dee0352c5e6b73/family1 2023-03-08 19:55:05,194 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-03-08 19:55:05,194 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 19:55:05,194 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(2446): Flush status journal for a526a5895c0e84ca75dee0352c5e6b73: 2023-03-08 19:55:05,195 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/e123da30ce2f41428dd443eea0aed6a8 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__uu7j4ls436n4ffo1jo9qs2bfi4f8e281f7t190358navj1pivng16ps7bbtjunvn/family1/e123da30ce2f41428dd443eea0aed6a8 2023-03-08 19:55:05,197 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__uu7j4ls436n4ffo1jo9qs2bfi4f8e281f7t190358navj1pivng16ps7bbtjunvn/family1/e123da30ce2f41428dd443eea0aed6a8 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/84b238084a224d8ba1a656d834f66924_SeqId_53_ 2023-03-08 19:55:05,198 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(849): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__uu7j4ls436n4ffo1jo9qs2bfi4f8e281f7t190358navj1pivng16ps7bbtjunvn/family1/e123da30ce2f41428dd443eea0aed6a8 into a526a5895c0e84ca75dee0352c5e6b73/family1 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/84b238084a224d8ba1a656d834f66924_SeqId_53_ - updating store file list. 2023-03-08 19:55:05,202 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(882): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/84b238084a224d8ba1a656d834f66924_SeqId_53_ into a526a5895c0e84ca75dee0352c5e6b73/family1 2023-03-08 19:55:05,202 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(855): Successfully loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__uu7j4ls436n4ffo1jo9qs2bfi4f8e281f7t190358navj1pivng16ps7bbtjunvn/family1/e123da30ce2f41428dd443eea0aed6a8 into a526a5895c0e84ca75dee0352c5e6b73/family1 (new location: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/84b238084a224d8ba1a656d834f66924_SeqId_53_) 2023-03-08 19:55:05,202 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__uu7j4ls436n4ffo1jo9qs2bfi4f8e281f7t190358navj1pivng16ps7bbtjunvn/family1/e123da30ce2f41428dd443eea0aed6a8 2023-03-08 19:55:05,204 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 19:55:05,205 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:55:05,206 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120376678335cf44ace9a51f6f0b243379a to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120376678335cf44ace9a51f6f0b243379a 2023-03-08 19:55:05,208 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128bba96cb95647424ebb5aa564e82b01b2 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128bba96cb95647424ebb5aa564e82b01b2 2023-03-08 19:55:05,208 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2 expired mob files are deleted 2023-03-08 19:55:05,208 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-08 19:55:05,208 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 19:55:05,208 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 19:55:05,208 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 19:55:05,208 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 19:55:05,209 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 19:55:05,210 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=64, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-08 19:55:05,211 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=64, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 901 msec 2023-03-08 19:55:05,218 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/ae6f273aeef4415585d3c817d2e83672, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/feb0948d958440778bf46f36b4590ee7, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/f09f89d84d194febbdd35590914ddd92_SeqId_46_] to archive 2023-03-08 19:55:05,219 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:55:05,221 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/ae6f273aeef4415585d3c817d2e83672 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/ae6f273aeef4415585d3c817d2e83672 2023-03-08 19:55:05,222 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/feb0948d958440778bf46f36b4590ee7 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/feb0948d958440778bf46f36b4590ee7 2023-03-08 19:55:05,224 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/f09f89d84d194febbdd35590914ddd92_SeqId_46_ to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/f09f89d84d194febbdd35590914ddd92_SeqId_46_ 2023-03-08 19:55:05,270 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'} 2023-03-08 19:55:05,271 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure2.ProcedureExecutor(1029): Stored pid=65, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:05,680 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-03-08 19:55:05,681 DEBUG [PEWorker-4] util.FSTableDescriptors(519): Deleted hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-03-08 19:55:05,681 INFO [PEWorker-4] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-03-08 19:55:05,683 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-03-08 19:55:05,685 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=a526a5895c0e84ca75dee0352c5e6b73, REOPEN/MOVE}] 2023-03-08 19:55:05,685 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=a526a5895c0e84ca75dee0352c5e6b73, REOPEN/MOVE 2023-03-08 19:55:05,686 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=a526a5895c0e84ca75dee0352c5e6b73, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:05,686 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678305305686"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678305305686"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678305305686"}]},"ts":"1678305305686"} 2023-03-08 19:55:05,687 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=68, ppid=67, state=RUNNABLE; CloseRegionProcedure a526a5895c0e84ca75dee0352c5e6b73, server=jenkins-hbase4.apache.org,33367,1678305218680}] 2023-03-08 19:55:05,839 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close a526a5895c0e84ca75dee0352c5e6b73 2023-03-08 19:55:05,839 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a526a5895c0e84ca75dee0352c5e6b73, disabling compactions & flushes 2023-03-08 19:55:05,839 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:55:05,839 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:55:05,839 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. after waiting 0 ms 2023-03-08 19:55:05,839 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:55:05,843 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=49 2023-03-08 19:55:05,843 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:55:05,843 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a526a5895c0e84ca75dee0352c5e6b73: 2023-03-08 19:55:05,843 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: a526a5895c0e84ca75dee0352c5e6b73 to self. 2023-03-08 19:55:05,845 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed a526a5895c0e84ca75dee0352c5e6b73 2023-03-08 19:55:05,845 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=a526a5895c0e84ca75dee0352c5e6b73, regionState=CLOSED 2023-03-08 19:55:05,845 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678305305845"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678305305845"}]},"ts":"1678305305845"} 2023-03-08 19:55:05,847 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=68, resume processing ppid=67 2023-03-08 19:55:05,847 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=68, ppid=67, state=SUCCESS; CloseRegionProcedure a526a5895c0e84ca75dee0352c5e6b73, server=jenkins-hbase4.apache.org,33367,1678305218680 in 159 msec 2023-03-08 19:55:05,848 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=a526a5895c0e84ca75dee0352c5e6b73, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,33367,1678305218680; forceNewPlan=false, retain=true 2023-03-08 19:55:05,998 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=a526a5895c0e84ca75dee0352c5e6b73, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:05,998 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678305305998"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678305305998"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678305305998"}]},"ts":"1678305305998"} 2023-03-08 19:55:06,000 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=69, ppid=67, state=RUNNABLE; OpenRegionProcedure a526a5895c0e84ca75dee0352c5e6b73, server=jenkins-hbase4.apache.org,33367,1678305218680}] 2023-03-08 19:55:06,154 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:55:06,154 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a526a5895c0e84ca75dee0352c5e6b73, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73.', STARTKEY => '', ENDKEY => ''} 2023-03-08 19:55:06,155 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily a526a5895c0e84ca75dee0352c5e6b73 2023-03-08 19:55:06,155 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:55:06,155 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a526a5895c0e84ca75dee0352c5e6b73 2023-03-08 19:55:06,155 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a526a5895c0e84ca75dee0352c5e6b73 2023-03-08 19:55:06,156 INFO [StoreOpener-a526a5895c0e84ca75dee0352c5e6b73-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a526a5895c0e84ca75dee0352c5e6b73 2023-03-08 19:55:06,157 DEBUG [StoreOpener-a526a5895c0e84ca75dee0352c5e6b73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1 2023-03-08 19:55:06,157 DEBUG [StoreOpener-a526a5895c0e84ca75dee0352c5e6b73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1 2023-03-08 19:55:06,157 INFO [StoreOpener-a526a5895c0e84ca75dee0352c5e6b73-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a526a5895c0e84ca75dee0352c5e6b73 columnFamilyName family1 2023-03-08 19:55:06,162 DEBUG [StoreOpener-a526a5895c0e84ca75dee0352c5e6b73-1] regionserver.HStore(539): loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/78dd1a89e85d4298a7abf8d79696fc24 2023-03-08 19:55:06,165 DEBUG [StoreOpener-a526a5895c0e84ca75dee0352c5e6b73-1] regionserver.HStore(539): loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/family1/84b238084a224d8ba1a656d834f66924_SeqId_53_ 2023-03-08 19:55:06,165 INFO [StoreOpener-a526a5895c0e84ca75dee0352c5e6b73-1] regionserver.HStore(310): Store=a526a5895c0e84ca75dee0352c5e6b73/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:55:06,166 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73 2023-03-08 19:55:06,167 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73 2023-03-08 19:55:06,170 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a526a5895c0e84ca75dee0352c5e6b73 2023-03-08 19:55:06,170 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a526a5895c0e84ca75dee0352c5e6b73; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72530460, jitterRate=0.08078807592391968}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 19:55:06,171 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a526a5895c0e84ca75dee0352c5e6b73: 2023-03-08 19:55:06,171 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73., pid=69, masterSystemTime=1678305306152 2023-03-08 19:55:06,173 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:55:06,173 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:55:06,173 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=a526a5895c0e84ca75dee0352c5e6b73, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:06,173 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678305306173"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678305306173"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678305306173"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678305306173"}]},"ts":"1678305306173"} 2023-03-08 19:55:06,176 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=69, resume processing ppid=67 2023-03-08 19:55:06,177 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=69, ppid=67, state=SUCCESS; OpenRegionProcedure a526a5895c0e84ca75dee0352c5e6b73, server=jenkins-hbase4.apache.org,33367,1678305218680 in 175 msec 2023-03-08 19:55:06,178 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=66 2023-03-08 19:55:06,178 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=66, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=a526a5895c0e84ca75dee0352c5e6b73, REOPEN/MOVE in 492 msec 2023-03-08 19:55:06,180 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-03-08 19:55:06,180 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 496 msec 2023-03-08 19:55:06,182 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 911 msec 2023-03-08 19:55:06,183 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=65 2023-03-08 19:55:06,184 INFO [Listener at localhost/34049] compactions.TestMobCompactor(278): alter status finished 2023-03-08 19:55:06,185 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39989] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-08 19:55:06,186 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] procedure2.ProcedureExecutor(1029): Stored pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-08 19:55:06,187 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-08 19:55:06,188 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 19:55:06,188 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=70, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678305906188 2023-03-08 19:55:06,188 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 19:55:06,189 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-03-08 19:55:06,189 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 19:55:06,189 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] zookeeper.ReadOnlyZKClient(139): Connect 0x5d9df324 to 127.0.0.1:56751 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 19:55:06,195 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@47f36016, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 19:55:06,195 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 19:55:06,195 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 19:55:06,195 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 19:55:06,195 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 19:55:06,195 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 19:55:06,195 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 19:55:06,195 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 19:55:06,195 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-08 19:55:06,195 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-08 19:55:06,195 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 19:55:06,196 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=70, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-08 19:55:06,197 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=70, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 10 msec 2023-03-08 19:55:06,243 INFO [Listener at localhost/34049] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=240 (was 243), OpenFileDescriptor=614 (was 613) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=163 (was 181), ProcessCount=170 (was 170), AvailableMemoryMB=8330 (was 8345) 2023-03-08 19:55:06,254 INFO [Listener at localhost/34049] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=240, OpenFileDescriptor=614, MaxFileDescriptor=60000, SystemLoadAverage=163, ProcessCount=170, AvailableMemoryMB=8330 2023-03-08 19:55:06,258 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-08 19:55:06,259 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure2.ProcedureExecutor(1029): Stored pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testGetAfterCompaction 2023-03-08 19:55:06,260 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 19:55:06,260 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testGetAfterCompaction" procId is: 71 2023-03-08 19:55:06,261 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 19:55:06,261 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-08 19:55:06,262 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0 2023-03-08 19:55:06,263 DEBUG [HFileArchiver-11] backup.HFileArchiver(153): Directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0 empty. 2023-03-08 19:55:06,263 DEBUG [HFileArchiver-11] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0 2023-03-08 19:55:06,264 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-03-08 19:55:06,362 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-08 19:55:06,563 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-08 19:55:06,672 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testGetAfterCompaction/.tabledesc/.tableinfo.0000000001 2023-03-08 19:55:06,674 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(7675): creating {ENCODED => de4401cd40cc1acb23343649dc4b9ac0, NAME => 'testGetAfterCompaction,,1678305306258.de4401cd40cc1acb23343649dc4b9ac0.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp 2023-03-08 19:55:06,864 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-08 19:55:07,080 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(866): Instantiated testGetAfterCompaction,,1678305306258.de4401cd40cc1acb23343649dc4b9ac0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:55:07,080 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1604): Closing de4401cd40cc1acb23343649dc4b9ac0, disabling compactions & flushes 2023-03-08 19:55:07,080 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1626): Closing region testGetAfterCompaction,,1678305306258.de4401cd40cc1acb23343649dc4b9ac0. 2023-03-08 19:55:07,080 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testGetAfterCompaction,,1678305306258.de4401cd40cc1acb23343649dc4b9ac0. 2023-03-08 19:55:07,080 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1714): Acquired close lock on testGetAfterCompaction,,1678305306258.de4401cd40cc1acb23343649dc4b9ac0. after waiting 0 ms 2023-03-08 19:55:07,080 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1724): Updates disabled for region testGetAfterCompaction,,1678305306258.de4401cd40cc1acb23343649dc4b9ac0. 2023-03-08 19:55:07,080 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1838): Closed testGetAfterCompaction,,1678305306258.de4401cd40cc1acb23343649dc4b9ac0. 2023-03-08 19:55:07,080 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1558): Region close journal for de4401cd40cc1acb23343649dc4b9ac0: 2023-03-08 19:55:07,082 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 19:55:07,083 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1678305306258.de4401cd40cc1acb23343649dc4b9ac0.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1678305307082"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678305307082"}]},"ts":"1678305307082"} 2023-03-08 19:55:07,084 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-08 19:55:07,084 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 19:55:07,085 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678305307084"}]},"ts":"1678305307084"} 2023-03-08 19:55:07,085 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLING in hbase:meta 2023-03-08 19:55:07,090 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=de4401cd40cc1acb23343649dc4b9ac0, ASSIGN}] 2023-03-08 19:55:07,091 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=de4401cd40cc1acb23343649dc4b9ac0, ASSIGN 2023-03-08 19:55:07,091 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testGetAfterCompaction, region=de4401cd40cc1acb23343649dc4b9ac0, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33367,1678305218680; forceNewPlan=false, retain=false 2023-03-08 19:55:07,242 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=de4401cd40cc1acb23343649dc4b9ac0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:07,243 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1678305306258.de4401cd40cc1acb23343649dc4b9ac0.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1678305307242"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678305307242"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678305307242"}]},"ts":"1678305307242"} 2023-03-08 19:55:07,244 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE; OpenRegionProcedure de4401cd40cc1acb23343649dc4b9ac0, server=jenkins-hbase4.apache.org,33367,1678305218680}] 2023-03-08 19:55:07,365 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-08 19:55:07,398 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testGetAfterCompaction,,1678305306258.de4401cd40cc1acb23343649dc4b9ac0. 2023-03-08 19:55:07,399 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => de4401cd40cc1acb23343649dc4b9ac0, NAME => 'testGetAfterCompaction,,1678305306258.de4401cd40cc1acb23343649dc4b9ac0.', STARTKEY => '', ENDKEY => ''} 2023-03-08 19:55:07,399 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro with path null and priority 1073741823 2023-03-08 19:55:07,400 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro from HTD of testGetAfterCompaction successfully. 2023-03-08 19:55:07,401 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testGetAfterCompaction de4401cd40cc1acb23343649dc4b9ac0 2023-03-08 19:55:07,401 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testGetAfterCompaction,,1678305306258.de4401cd40cc1acb23343649dc4b9ac0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:55:07,401 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for de4401cd40cc1acb23343649dc4b9ac0 2023-03-08 19:55:07,401 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for de4401cd40cc1acb23343649dc4b9ac0 2023-03-08 19:55:07,402 INFO [StoreOpener-de4401cd40cc1acb23343649dc4b9ac0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region de4401cd40cc1acb23343649dc4b9ac0 2023-03-08 19:55:07,404 DEBUG [StoreOpener-de4401cd40cc1acb23343649dc4b9ac0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1 2023-03-08 19:55:07,404 DEBUG [StoreOpener-de4401cd40cc1acb23343649dc4b9ac0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1 2023-03-08 19:55:07,404 INFO [StoreOpener-de4401cd40cc1acb23343649dc4b9ac0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region de4401cd40cc1acb23343649dc4b9ac0 columnFamilyName f1 2023-03-08 19:55:07,405 INFO [StoreOpener-de4401cd40cc1acb23343649dc4b9ac0-1] regionserver.HStore(310): Store=de4401cd40cc1acb23343649dc4b9ac0/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:55:07,406 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0 2023-03-08 19:55:07,407 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0 2023-03-08 19:55:07,409 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for de4401cd40cc1acb23343649dc4b9ac0 2023-03-08 19:55:07,410 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 19:55:07,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened de4401cd40cc1acb23343649dc4b9ac0; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69874466, jitterRate=0.041210681200027466}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 19:55:07,411 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for de4401cd40cc1acb23343649dc4b9ac0: 2023-03-08 19:55:07,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testGetAfterCompaction,,1678305306258.de4401cd40cc1acb23343649dc4b9ac0., pid=73, masterSystemTime=1678305307396 2023-03-08 19:55:07,412 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testGetAfterCompaction,,1678305306258.de4401cd40cc1acb23343649dc4b9ac0. 2023-03-08 19:55:07,413 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testGetAfterCompaction,,1678305306258.de4401cd40cc1acb23343649dc4b9ac0. 2023-03-08 19:55:07,413 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=de4401cd40cc1acb23343649dc4b9ac0, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:07,413 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testGetAfterCompaction,,1678305306258.de4401cd40cc1acb23343649dc4b9ac0.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1678305307413"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678305307413"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678305307413"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678305307413"}]},"ts":"1678305307413"} 2023-03-08 19:55:07,415 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-03-08 19:55:07,415 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; OpenRegionProcedure de4401cd40cc1acb23343649dc4b9ac0, server=jenkins-hbase4.apache.org,33367,1678305218680 in 170 msec 2023-03-08 19:55:07,416 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=72, resume processing ppid=71 2023-03-08 19:55:07,416 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=72, ppid=71, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=de4401cd40cc1acb23343649dc4b9ac0, ASSIGN in 325 msec 2023-03-08 19:55:07,417 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 19:55:07,417 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678305307417"}]},"ts":"1678305307417"} 2023-03-08 19:55:07,418 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLED in hbase:meta 2023-03-08 19:55:07,420 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 19:55:07,421 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=71, state=SUCCESS; CreateTableProcedure table=testGetAfterCompaction in 1.1620 sec 2023-03-08 19:55:07,659 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-03-08 19:55:07,659 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro Metrics about HBase RegionObservers 2023-03-08 19:55:08,366 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-08 19:55:08,366 INFO [Listener at localhost/34049] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testGetAfterCompaction, procId: 71 completed 2023-03-08 19:55:08,367 DEBUG [Listener at localhost/34049] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testGetAfterCompaction get assigned. Timeout = 60000ms 2023-03-08 19:55:08,368 INFO [Listener at localhost/34049] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-03-08 19:55:08,371 INFO [Listener at localhost/34049] hbase.HBaseTestingUtility(3484): All regions for table testGetAfterCompaction assigned to meta. Checking AM states. 2023-03-08 19:55:08,372 INFO [Listener at localhost/34049] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-03-08 19:55:08,372 INFO [Listener at localhost/34049] hbase.HBaseTestingUtility(3504): All regions for table testGetAfterCompaction assigned. 2023-03-08 19:55:08,376 INFO [Listener at localhost/34049] regionserver.HRegion(2745): Flushing de4401cd40cc1acb23343649dc4b9ac0 1/1 column families, dataSize=37 B heapSize=368 B 2023-03-08 19:55:08,388 INFO [Listener at localhost/34049] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308a99961299cbe479fa2d61f0dcf764cab to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230308a99961299cbe479fa2d61f0dcf764cab 2023-03-08 19:55:08,793 INFO [Listener at localhost/34049] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/.tmp/f1/23294267266e4104a91ae27652607813 2023-03-08 19:55:08,798 DEBUG [Listener at localhost/34049] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/.tmp/f1/23294267266e4104a91ae27652607813 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/23294267266e4104a91ae27652607813 2023-03-08 19:55:08,801 INFO [Listener at localhost/34049] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/23294267266e4104a91ae27652607813, entries=1, sequenceid=5, filesize=4.9 K 2023-03-08 19:55:08,802 INFO [Listener at localhost/34049] regionserver.HRegion(2948): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for de4401cd40cc1acb23343649dc4b9ac0 in 426ms, sequenceid=5, compaction requested=false 2023-03-08 19:55:08,802 DEBUG [Listener at localhost/34049] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testGetAfterCompaction' 2023-03-08 19:55:08,803 DEBUG [Listener at localhost/34049] regionserver.HRegion(2446): Flush status journal for de4401cd40cc1acb23343649dc4b9ac0: 2023-03-08 19:55:08,804 INFO [Listener at localhost/34049] regionserver.HRegion(2745): Flushing de4401cd40cc1acb23343649dc4b9ac0 1/1 column families, dataSize=37 B heapSize=368 B 2023-03-08 19:55:09,217 INFO [Listener at localhost/34049] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308699fa6ff7abd426e965146adcfcb1dae to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230308699fa6ff7abd426e965146adcfcb1dae 2023-03-08 19:55:09,223 INFO [Listener at localhost/34049] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/.tmp/f1/d7e5449674104731992bee1c87095d27 2023-03-08 19:55:09,227 DEBUG [Listener at localhost/34049] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/.tmp/f1/d7e5449674104731992bee1c87095d27 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/d7e5449674104731992bee1c87095d27 2023-03-08 19:55:09,231 INFO [Listener at localhost/34049] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/d7e5449674104731992bee1c87095d27, entries=1, sequenceid=9, filesize=4.9 K 2023-03-08 19:55:09,232 INFO [Listener at localhost/34049] regionserver.HRegion(2948): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for de4401cd40cc1acb23343649dc4b9ac0 in 428ms, sequenceid=9, compaction requested=false 2023-03-08 19:55:09,232 DEBUG [Listener at localhost/34049] regionserver.HRegion(2446): Flush status journal for de4401cd40cc1acb23343649dc4b9ac0: 2023-03-08 19:55:09,233 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39989] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testGetAfterCompaction 2023-03-08 19:55:09,235 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] procedure2.ProcedureExecutor(1029): Stored pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-03-08 19:55:09,235 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-03-08 19:55:09,236 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] mob.MobUtils(833): start MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 19:55:09,236 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=74, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678305909236 2023-03-08 19:55:09,237 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-08 19:55:09,237 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testGetAfterCompaction' and column 'f1' 2023-03-08 19:55:09,237 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testGetAfterCompaction' column='f1' 2023-03-08 19:55:09,237 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] zookeeper.ReadOnlyZKClient(139): Connect 0x12ee7b19 to 127.0.0.1:56751 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 19:55:09,247 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4ec14a82, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 19:55:09,247 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230308 for table='testGetAfterCompaction' and column='f1' 2023-03-08 19:55:09,285 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/23294267266e4104a91ae27652607813; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305308793; access_time=1678305308378; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:09,285 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/d7e5449674104731992bee1c87095d27; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305309222; access_time=1678305308806; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:09,336 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/23294267266e4104a91ae27652607813; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305308793; access_time=1678305308378; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:09,336 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/d7e5449674104731992bee1c87095d27; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305309222; access_time=1678305308806; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:09,387 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/23294267266e4104a91ae27652607813; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305308793; access_time=1678305308378; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:09,387 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/d7e5449674104731992bee1c87095d27; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305309222; access_time=1678305308806; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:09,438 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/23294267266e4104a91ae27652607813; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305308793; access_time=1678305308378; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:09,438 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/d7e5449674104731992bee1c87095d27; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305309222; access_time=1678305308806; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:09,489 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/23294267266e4104a91ae27652607813; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305308793; access_time=1678305308378; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:09,489 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/d7e5449674104731992bee1c87095d27; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305309222; access_time=1678305308806; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:09,501 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 19:55:09,540 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/23294267266e4104a91ae27652607813; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305308793; access_time=1678305308378; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:09,540 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/d7e5449674104731992bee1c87095d27; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305309222; access_time=1678305308806; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:09,591 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/23294267266e4104a91ae27652607813; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305308793; access_time=1678305308378; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:09,591 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/d7e5449674104731992bee1c87095d27; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305309222; access_time=1678305308806; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:09,641 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/23294267266e4104a91ae27652607813; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305308793; access_time=1678305308378; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:09,642 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/d7e5449674104731992bee1c87095d27; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305309222; access_time=1678305308806; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:09,692 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/23294267266e4104a91ae27652607813; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305308793; access_time=1678305308378; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:09,692 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/d7e5449674104731992bee1c87095d27; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305309222; access_time=1678305308806; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:09,743 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/23294267266e4104a91ae27652607813; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305308793; access_time=1678305308378; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:09,744 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/d7e5449674104731992bee1c87095d27; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305309222; access_time=1678305308806; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:09,794 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/23294267266e4104a91ae27652607813; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305308793; access_time=1678305308378; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:09,794 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/d7e5449674104731992bee1c87095d27; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305309222; access_time=1678305308806; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:09,845 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/23294267266e4104a91ae27652607813; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305308793; access_time=1678305308378; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:09,845 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/d7e5449674104731992bee1c87095d27; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305309222; access_time=1678305308806; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:09,896 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/23294267266e4104a91ae27652607813; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305308793; access_time=1678305308378; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:09,896 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/d7e5449674104731992bee1c87095d27; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305309222; access_time=1678305308806; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:09,947 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/23294267266e4104a91ae27652607813; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305308793; access_time=1678305308378; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:09,947 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/d7e5449674104731992bee1c87095d27; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305309222; access_time=1678305308806; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:09,998 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/23294267266e4104a91ae27652607813; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305308793; access_time=1678305308378; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:09,998 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/d7e5449674104731992bee1c87095d27; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305309222; access_time=1678305308806; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:10,049 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/23294267266e4104a91ae27652607813; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305308793; access_time=1678305308378; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:10,049 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/d7e5449674104731992bee1c87095d27; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305309222; access_time=1678305308806; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:10,070 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308fc790f7d82e845ad9d7af79e3a0d9b0a to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230308fc790f7d82e845ad9d7af79e3a0d9b0a 2023-03-08 19:55:10,071 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230308 table='testGetAfterCompaction' column='f1' 2023-03-08 19:55:10,084 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 19:55:10,085 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33104, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 19:55:10,090 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 19:55:10,092 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43468, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 19:55:10,099 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/23294267266e4104a91ae27652607813; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305308793; access_time=1678305308378; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:10,100 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/d7e5449674104731992bee1c87095d27; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305309222; access_time=1678305308806; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:10,100 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230308/f1/05ab60c01faa459285b7086be67c9f2c first=Optional[r1] last=Optional[r2] 2023-03-08 19:55:10,100 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testGetAfterCompaction,,1678305306258.de4401cd40cc1acb23343649dc4b9ac0., hostname=jenkins-hbase4.apache.org,33367,1678305218680, seqNum=2 for row with hfile group [{f1,hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230308/f1/05ab60c01faa459285b7086be67c9f2c}] 2023-03-08 19:55:10,105 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(763): Validating hfile at hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230308/f1/05ab60c01faa459285b7086be67c9f2c for inclusion in de4401cd40cc1acb23343649dc4b9ac0/f1 2023-03-08 19:55:10,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-03-08 19:55:10,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 19:55:10,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(2446): Flush status journal for de4401cd40cc1acb23343649dc4b9ac0: 2023-03-08 19:55:10,109 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230308/f1/05ab60c01faa459285b7086be67c9f2c to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testGetAfterCompaction__c0f0vmmuj9o7c73bru3gcrs1ct67v7g1v9hgngpjd41i455v9aabqlikclmp2ncr/f1/05ab60c01faa459285b7086be67c9f2c 2023-03-08 19:55:10,111 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testGetAfterCompaction__c0f0vmmuj9o7c73bru3gcrs1ct67v7g1v9hgngpjd41i455v9aabqlikclmp2ncr/f1/05ab60c01faa459285b7086be67c9f2c as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/1fbdd1ba35ac431cb62fa51920aaf722_SeqId_12_ 2023-03-08 19:55:10,112 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(849): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testGetAfterCompaction__c0f0vmmuj9o7c73bru3gcrs1ct67v7g1v9hgngpjd41i455v9aabqlikclmp2ncr/f1/05ab60c01faa459285b7086be67c9f2c into de4401cd40cc1acb23343649dc4b9ac0/f1 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/1fbdd1ba35ac431cb62fa51920aaf722_SeqId_12_ - updating store file list. 2023-03-08 19:55:10,115 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(882): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/1fbdd1ba35ac431cb62fa51920aaf722_SeqId_12_ into de4401cd40cc1acb23343649dc4b9ac0/f1 2023-03-08 19:55:10,115 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(855): Successfully loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testGetAfterCompaction__c0f0vmmuj9o7c73bru3gcrs1ct67v7g1v9hgngpjd41i455v9aabqlikclmp2ncr/f1/05ab60c01faa459285b7086be67c9f2c into de4401cd40cc1acb23343649dc4b9ac0/f1 (new location: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/1fbdd1ba35ac431cb62fa51920aaf722_SeqId_12_) 2023-03-08 19:55:10,116 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testGetAfterCompaction__c0f0vmmuj9o7c73bru3gcrs1ct67v7g1v9hgngpjd41i455v9aabqlikclmp2ncr/f1/05ab60c01faa459285b7086be67c9f2c 2023-03-08 19:55:10,116 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(1912): de4401cd40cc1acb23343649dc4b9ac0/f1 is initiating minor compaction 2023-03-08 19:55:10,116 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@f885d49; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 19:55:10,116 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of de4401cd40cc1acb23343649dc4b9ac0/f1 in testGetAfterCompaction,,1678305306258.de4401cd40cc1acb23343649dc4b9ac0. 2023-03-08 19:55:10,116 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/d7e5449674104731992bee1c87095d27, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/1fbdd1ba35ac431cb62fa51920aaf722_SeqId_12_] into tmpdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/.tmp, totalSize=10.0 K 2023-03-08 19:55:10,117 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting d7e5449674104731992bee1c87095d27, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-08 19:55:10,117 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 1fbdd1ba35ac431cb62fa51920aaf722_SeqId_12_, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-08 19:55:10,118 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230308 table='testGetAfterCompaction' column='f1' 2023-03-08 19:55:10,119 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:55:10,120 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230308699fa6ff7abd426e965146adcfcb1dae to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230308699fa6ff7abd426e965146adcfcb1dae 2023-03-08 19:55:10,121 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230308a99961299cbe479fa2d61f0dcf764cab to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230308a99961299cbe479fa2d61f0dcf764cab 2023-03-08 19:55:10,122 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testGetAfterCompaction 2 expired mob files are deleted 2023-03-08 19:55:10,122 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230308' 2023-03-08 19:55:10,122 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230308 for table='testGetAfterCompaction' and column='f1' 2023-03-08 19:55:10,122 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testGetAfterCompaction' column='f1' 2023-03-08 19:55:10,122 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] mob.MobUtils(840): end MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 19:55:10,123 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=74, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-03-08 19:55:10,124 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] throttle.PressureAwareThroughputController(145): de4401cd40cc1acb23343649dc4b9ac0#f1#compaction#77 average throughput is 0.13 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 19:55:10,125 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=74, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE in 890 msec 2023-03-08 19:55:10,150 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/1fbdd1ba35ac431cb62fa51920aaf722_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678305310066; access_time=1678305309258; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 19:55:10,150 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/23294267266e4104a91ae27652607813; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305308793; access_time=1678305308378; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:10,151 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/d7e5449674104731992bee1c87095d27; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305309222; access_time=1678305308806; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:10,154 DEBUG [Listener at localhost/34049] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-03-08 19:55:10,158 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33110, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-03-08 19:55:10,158 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33367] regionserver.RSRpcServices(1662): Compacting testGetAfterCompaction,,1678305306258.de4401cd40cc1acb23343649dc4b9ac0. 2023-03-08 19:55:10,159 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33367] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 2 compacting, 0 eligible, 16 blocking 2023-03-08 19:55:10,159 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33367] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-03-08 19:55:10,159 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33367] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-03-08 19:55:10,159 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33367] regionserver.CompactSplit(415): Not compacting testGetAfterCompaction,,1678305306258.de4401cd40cc1acb23343649dc4b9ac0. because compaction request was cancelled 2023-03-08 19:55:10,211 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/1fbdd1ba35ac431cb62fa51920aaf722_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678305310066; access_time=1678305309258; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 19:55:10,211 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/23294267266e4104a91ae27652607813; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305308793; access_time=1678305308378; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:10,211 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/d7e5449674104731992bee1c87095d27; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305309222; access_time=1678305308806; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:10,262 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/1fbdd1ba35ac431cb62fa51920aaf722_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678305310066; access_time=1678305309258; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 19:55:10,262 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/23294267266e4104a91ae27652607813; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305308793; access_time=1678305308378; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:10,262 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/d7e5449674104731992bee1c87095d27; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305309222; access_time=1678305308806; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:10,313 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/1fbdd1ba35ac431cb62fa51920aaf722_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678305310066; access_time=1678305309258; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 19:55:10,313 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/23294267266e4104a91ae27652607813; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305308793; access_time=1678305308378; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:10,313 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/d7e5449674104731992bee1c87095d27; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305309222; access_time=1678305308806; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:10,364 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/1fbdd1ba35ac431cb62fa51920aaf722_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678305310066; access_time=1678305309258; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 19:55:10,364 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/23294267266e4104a91ae27652607813; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305308793; access_time=1678305308378; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:10,365 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/d7e5449674104731992bee1c87095d27; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305309222; access_time=1678305308806; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:10,415 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/1fbdd1ba35ac431cb62fa51920aaf722_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678305310066; access_time=1678305309258; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 19:55:10,416 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/23294267266e4104a91ae27652607813; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305308793; access_time=1678305308378; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:10,416 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/d7e5449674104731992bee1c87095d27; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305309222; access_time=1678305308806; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:10,467 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/1fbdd1ba35ac431cb62fa51920aaf722_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678305310066; access_time=1678305309258; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 19:55:10,467 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/23294267266e4104a91ae27652607813; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305308793; access_time=1678305308378; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:10,467 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/d7e5449674104731992bee1c87095d27; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305309222; access_time=1678305308806; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:10,518 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/1fbdd1ba35ac431cb62fa51920aaf722_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678305310066; access_time=1678305309258; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 19:55:10,518 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/23294267266e4104a91ae27652607813; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305308793; access_time=1678305308378; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:10,518 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/d7e5449674104731992bee1c87095d27; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305309222; access_time=1678305308806; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:10,569 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/1fbdd1ba35ac431cb62fa51920aaf722_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678305310066; access_time=1678305309258; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 19:55:10,569 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/23294267266e4104a91ae27652607813; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305308793; access_time=1678305308378; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:10,569 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/d7e5449674104731992bee1c87095d27; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305309222; access_time=1678305308806; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:10,620 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/1fbdd1ba35ac431cb62fa51920aaf722_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678305310066; access_time=1678305309258; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 19:55:10,620 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/23294267266e4104a91ae27652607813; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305308793; access_time=1678305308378; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:10,620 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/d7e5449674104731992bee1c87095d27; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305309222; access_time=1678305308806; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:10,671 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/1fbdd1ba35ac431cb62fa51920aaf722_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678305310066; access_time=1678305309258; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 19:55:10,671 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/23294267266e4104a91ae27652607813; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305308793; access_time=1678305308378; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:10,671 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/d7e5449674104731992bee1c87095d27; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305309222; access_time=1678305308806; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:10,722 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/1fbdd1ba35ac431cb62fa51920aaf722_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678305310066; access_time=1678305309258; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 19:55:10,722 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/23294267266e4104a91ae27652607813; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305308793; access_time=1678305308378; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:10,722 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/d7e5449674104731992bee1c87095d27; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305309222; access_time=1678305308806; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:10,773 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/1fbdd1ba35ac431cb62fa51920aaf722_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678305310066; access_time=1678305309258; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 19:55:10,773 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/23294267266e4104a91ae27652607813; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305308793; access_time=1678305308378; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:10,773 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/d7e5449674104731992bee1c87095d27; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305309222; access_time=1678305308806; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:10,824 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/1fbdd1ba35ac431cb62fa51920aaf722_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678305310066; access_time=1678305309258; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 19:55:10,824 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/23294267266e4104a91ae27652607813; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305308793; access_time=1678305308378; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:10,824 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/d7e5449674104731992bee1c87095d27; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305309222; access_time=1678305308806; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:10,875 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/1fbdd1ba35ac431cb62fa51920aaf722_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678305310066; access_time=1678305309258; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 19:55:10,875 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/23294267266e4104a91ae27652607813; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305308793; access_time=1678305308378; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:10,875 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/d7e5449674104731992bee1c87095d27; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305309222; access_time=1678305308806; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:10,926 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/1fbdd1ba35ac431cb62fa51920aaf722_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678305310066; access_time=1678305309258; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 19:55:10,926 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/23294267266e4104a91ae27652607813; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305308793; access_time=1678305308378; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:10,926 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/d7e5449674104731992bee1c87095d27; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305309222; access_time=1678305308806; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:10,937 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/.tmp/f1/7dc9c766eb534cda933aad38ece09ac1 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/7dc9c766eb534cda933aad38ece09ac1 2023-03-08 19:55:10,941 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 2 file(s) in de4401cd40cc1acb23343649dc4b9ac0/f1 of de4401cd40cc1acb23343649dc4b9ac0 into 7dc9c766eb534cda933aad38ece09ac1(size=5.1 K), total size for store is 10.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 19:55:10,942 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for de4401cd40cc1acb23343649dc4b9ac0: 2023-03-08 19:55:10,942 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testGetAfterCompaction,,1678305306258.de4401cd40cc1acb23343649dc4b9ac0., storeName=f1, fileCount=2, fileSize=10.0 K (4.9 K, 5.1 K), priority=2, time=1678305310116; duration=0sec 2023-03-08 19:55:10,942 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 19:55:10,976 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/1fbdd1ba35ac431cb62fa51920aaf722_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678305310066; access_time=1678305309258; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-08 19:55:10,977 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/23294267266e4104a91ae27652607813; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305308793; access_time=1678305308378; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:10,977 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/7dc9c766eb534cda933aad38ece09ac1; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1678305310933; access_time=1678305310121; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:10,977 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/d7e5449674104731992bee1c87095d27; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305309222; access_time=1678305308806; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:11,017 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/d7e5449674104731992bee1c87095d27, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/1fbdd1ba35ac431cb62fa51920aaf722_SeqId_12_] to archive 2023-03-08 19:55:11,018 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:55:11,019 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/d7e5449674104731992bee1c87095d27 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/d7e5449674104731992bee1c87095d27 2023-03-08 19:55:11,021 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/1fbdd1ba35ac431cb62fa51920aaf722_SeqId_12_ to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/1fbdd1ba35ac431cb62fa51920aaf722_SeqId_12_ 2023-03-08 19:55:11,027 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/23294267266e4104a91ae27652607813; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678305308793; access_time=1678305308378; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:11,027 INFO [Listener at localhost/34049] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/7dc9c766eb534cda933aad38ece09ac1; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1678305310933; access_time=1678305310121; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-08 19:55:11,142 INFO [Listener at localhost/34049] client.HBaseAdmin$15(890): Started disable of testGetAfterCompaction 2023-03-08 19:55:11,148 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testGetAfterCompaction 2023-03-08 19:55:11,153 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure2.ProcedureExecutor(1029): Stored pid=75, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testGetAfterCompaction 2023-03-08 19:55:11,156 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678305311156"}]},"ts":"1678305311156"} 2023-03-08 19:55:11,157 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-03-08 19:55:11,158 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLING in hbase:meta 2023-03-08 19:55:11,159 INFO [PEWorker-5] procedure.DisableTableProcedure(293): Set testGetAfterCompaction to state=DISABLING 2023-03-08 19:55:11,161 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=de4401cd40cc1acb23343649dc4b9ac0, UNASSIGN}] 2023-03-08 19:55:11,162 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=de4401cd40cc1acb23343649dc4b9ac0, UNASSIGN 2023-03-08 19:55:11,163 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=de4401cd40cc1acb23343649dc4b9ac0, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:11,163 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1678305306258.de4401cd40cc1acb23343649dc4b9ac0.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1678305311163"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678305311163"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678305311163"}]},"ts":"1678305311163"} 2023-03-08 19:55:11,164 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; CloseRegionProcedure de4401cd40cc1acb23343649dc4b9ac0, server=jenkins-hbase4.apache.org,33367,1678305218680}] 2023-03-08 19:55:11,258 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-03-08 19:55:11,316 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close de4401cd40cc1acb23343649dc4b9ac0 2023-03-08 19:55:11,317 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing de4401cd40cc1acb23343649dc4b9ac0, disabling compactions & flushes 2023-03-08 19:55:11,317 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testGetAfterCompaction,,1678305306258.de4401cd40cc1acb23343649dc4b9ac0. 2023-03-08 19:55:11,317 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testGetAfterCompaction,,1678305306258.de4401cd40cc1acb23343649dc4b9ac0. 2023-03-08 19:55:11,317 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testGetAfterCompaction,,1678305306258.de4401cd40cc1acb23343649dc4b9ac0. after waiting 0 ms 2023-03-08 19:55:11,317 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testGetAfterCompaction,,1678305306258.de4401cd40cc1acb23343649dc4b9ac0. 2023-03-08 19:55:11,321 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-03-08 19:55:11,321 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-03-08 19:55:11,323 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testGetAfterCompaction,,1678305306258.de4401cd40cc1acb23343649dc4b9ac0. 2023-03-08 19:55:11,323 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for de4401cd40cc1acb23343649dc4b9ac0: 2023-03-08 19:55:11,324 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed de4401cd40cc1acb23343649dc4b9ac0 2023-03-08 19:55:11,324 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=de4401cd40cc1acb23343649dc4b9ac0, regionState=CLOSED 2023-03-08 19:55:11,325 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1678305306258.de4401cd40cc1acb23343649dc4b9ac0.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1678305311324"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678305311324"}]},"ts":"1678305311324"} 2023-03-08 19:55:11,327 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=76 2023-03-08 19:55:11,327 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=76, state=SUCCESS; CloseRegionProcedure de4401cd40cc1acb23343649dc4b9ac0, server=jenkins-hbase4.apache.org,33367,1678305218680 in 162 msec 2023-03-08 19:55:11,327 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=75 2023-03-08 19:55:11,328 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=75, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=de4401cd40cc1acb23343649dc4b9ac0, UNASSIGN in 166 msec 2023-03-08 19:55:11,328 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678305311328"}]},"ts":"1678305311328"} 2023-03-08 19:55:11,329 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLED in hbase:meta 2023-03-08 19:55:11,335 INFO [PEWorker-5] procedure.DisableTableProcedure(305): Set testGetAfterCompaction to state=DISABLED 2023-03-08 19:55:11,336 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=75, state=SUCCESS; DisableTableProcedure table=testGetAfterCompaction in 186 msec 2023-03-08 19:55:11,459 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-03-08 19:55:11,459 INFO [Listener at localhost/34049] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testGetAfterCompaction, procId: 75 completed 2023-03-08 19:55:11,464 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testGetAfterCompaction 2023-03-08 19:55:11,469 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure2.ProcedureExecutor(1029): Stored pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testGetAfterCompaction 2023-03-08 19:55:11,471 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-08 19:55:11,472 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=78, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-08 19:55:11,473 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-03-08 19:55:11,475 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0 2023-03-08 19:55:11,477 DEBUG [HFileArchiver-12] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1, FileablePath, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/recovered.edits] 2023-03-08 19:55:11,480 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/23294267266e4104a91ae27652607813 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/23294267266e4104a91ae27652607813 2023-03-08 19:55:11,482 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/7dc9c766eb534cda933aad38ece09ac1 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/f1/7dc9c766eb534cda933aad38ece09ac1 2023-03-08 19:55:11,484 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/recovered.edits/15.seqid to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0/recovered.edits/15.seqid 2023-03-08 19:55:11,485 DEBUG [HFileArchiver-12] backup.HFileArchiver(596): Deleted hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testGetAfterCompaction/de4401cd40cc1acb23343649dc4b9ac0 2023-03-08 19:55:11,485 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-03-08 19:55:11,485 DEBUG [PEWorker-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-03-08 19:55:11,486 DEBUG [PEWorker-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1] 2023-03-08 19:55:11,488 DEBUG [PEWorker-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230308fc790f7d82e845ad9d7af79e3a0d9b0a to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230308fc790f7d82e845ad9d7af79e3a0d9b0a 2023-03-08 19:55:11,489 DEBUG [PEWorker-2] backup.HFileArchiver(596): Deleted hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-03-08 19:55:11,491 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=78, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-08 19:55:11,513 WARN [PEWorker-2] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testGetAfterCompaction from hbase:meta 2023-03-08 19:55:11,516 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(421): Removing 'testGetAfterCompaction' descriptor. 2023-03-08 19:55:11,517 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=78, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-08 19:55:11,517 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(411): Removing 'testGetAfterCompaction' from region states. 2023-03-08 19:55:11,517 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction,,1678305306258.de4401cd40cc1acb23343649dc4b9ac0.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1678305311517"}]},"ts":"9223372036854775807"} 2023-03-08 19:55:11,519 INFO [PEWorker-2] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-03-08 19:55:11,519 DEBUG [PEWorker-2] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => de4401cd40cc1acb23343649dc4b9ac0, NAME => 'testGetAfterCompaction,,1678305306258.de4401cd40cc1acb23343649dc4b9ac0.', STARTKEY => '', ENDKEY => ''}] 2023-03-08 19:55:11,519 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(415): Marking 'testGetAfterCompaction' as deleted. 2023-03-08 19:55:11,519 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1678305311519"}]},"ts":"9223372036854775807"} 2023-03-08 19:55:11,521 INFO [PEWorker-2] hbase.MetaTableAccessor(1658): Deleted table testGetAfterCompaction state from META 2023-03-08 19:55:11,522 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(130): Finished pid=78, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-08 19:55:11,523 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=78, state=SUCCESS; DeleteTableProcedure table=testGetAfterCompaction in 57 msec 2023-03-08 19:55:11,574 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-03-08 19:55:11,575 INFO [Listener at localhost/34049] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testGetAfterCompaction, procId: 78 completed 2023-03-08 19:55:11,589 INFO [Listener at localhost/34049] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=244 (was 240) - Thread LEAK? -, OpenFileDescriptor=620 (was 614) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=158 (was 163), ProcessCount=170 (was 170), AvailableMemoryMB=8326 (was 8330) 2023-03-08 19:55:11,599 INFO [Listener at localhost/34049] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=244, OpenFileDescriptor=620, MaxFileDescriptor=60000, SystemLoadAverage=158, ProcessCount=170, AvailableMemoryMB=8326 2023-03-08 19:55:11,601 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-08 19:55:11,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testCompactionWithHFileLink 2023-03-08 19:55:11,603 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 19:55:11,603 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testCompactionWithHFileLink" procId is: 79 2023-03-08 19:55:11,604 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 19:55:11,604 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-08 19:55:11,605 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd 2023-03-08 19:55:11,605 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb 2023-03-08 19:55:11,605 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8 2023-03-08 19:55:11,606 DEBUG [HFileArchiver-13] backup.HFileArchiver(153): Directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd empty. 2023-03-08 19:55:11,606 DEBUG [HFileArchiver-15] backup.HFileArchiver(153): Directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8 empty. 2023-03-08 19:55:11,606 DEBUG [HFileArchiver-14] backup.HFileArchiver(153): Directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb empty. 2023-03-08 19:55:11,606 DEBUG [HFileArchiver-13] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd 2023-03-08 19:55:11,606 DEBUG [HFileArchiver-14] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb 2023-03-08 19:55:11,606 DEBUG [HFileArchiver-15] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8 2023-03-08 19:55:11,607 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testCompactionWithHFileLink regions 2023-03-08 19:55:11,705 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-08 19:55:11,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-08 19:55:12,015 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-03-08 19:55:12,016 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(7675): creating {ENCODED => bf25ef5682cfde9aa90a5d6fe8229bdd, NAME => 'testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp 2023-03-08 19:55:12,016 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(7675): creating {ENCODED => d392b5a609f9661af565e4c73dab47f8, NAME => 'testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp 2023-03-08 19:55:12,017 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(7675): creating {ENCODED => 9c99479945efcf910b16288e607c83eb, NAME => 'testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp 2023-03-08 19:55:12,028 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:55:12,028 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1604): Closing d392b5a609f9661af565e4c73dab47f8, disabling compactions & flushes 2023-03-08 19:55:12,028 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. 2023-03-08 19:55:12,028 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. 2023-03-08 19:55:12,028 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. after waiting 0 ms 2023-03-08 19:55:12,028 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. 2023-03-08 19:55:12,028 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. 2023-03-08 19:55:12,028 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1558): Region close journal for d392b5a609f9661af565e4c73dab47f8: 2023-03-08 19:55:12,207 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-08 19:55:12,427 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:55:12,427 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:55:12,428 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1604): Closing bf25ef5682cfde9aa90a5d6fe8229bdd, disabling compactions & flushes 2023-03-08 19:55:12,428 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1604): Closing 9c99479945efcf910b16288e607c83eb, disabling compactions & flushes 2023-03-08 19:55:12,428 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. 2023-03-08 19:55:12,428 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. 2023-03-08 19:55:12,428 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. 2023-03-08 19:55:12,428 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. 2023-03-08 19:55:12,428 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. after waiting 0 ms 2023-03-08 19:55:12,428 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. 2023-03-08 19:55:12,428 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. after waiting 0 ms 2023-03-08 19:55:12,428 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. 2023-03-08 19:55:12,428 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. 2023-03-08 19:55:12,428 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. 2023-03-08 19:55:12,428 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1558): Region close journal for bf25ef5682cfde9aa90a5d6fe8229bdd: 2023-03-08 19:55:12,428 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1558): Region close journal for 9c99479945efcf910b16288e607c83eb: 2023-03-08 19:55:12,430 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 19:55:12,431 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678305312431"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678305312431"}]},"ts":"1678305312431"} 2023-03-08 19:55:12,431 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678305312431"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678305312431"}]},"ts":"1678305312431"} 2023-03-08 19:55:12,431 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678305312431"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678305312431"}]},"ts":"1678305312431"} 2023-03-08 19:55:12,433 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-08 19:55:12,433 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 19:55:12,434 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678305312434"}]},"ts":"1678305312434"} 2023-03-08 19:55:12,435 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-03-08 19:55:12,440 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bf25ef5682cfde9aa90a5d6fe8229bdd, ASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=9c99479945efcf910b16288e607c83eb, ASSIGN}, {pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d392b5a609f9661af565e4c73dab47f8, ASSIGN}] 2023-03-08 19:55:12,442 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=9c99479945efcf910b16288e607c83eb, ASSIGN 2023-03-08 19:55:12,442 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bf25ef5682cfde9aa90a5d6fe8229bdd, ASSIGN 2023-03-08 19:55:12,442 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d392b5a609f9661af565e4c73dab47f8, ASSIGN 2023-03-08 19:55:12,442 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=9c99479945efcf910b16288e607c83eb, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33367,1678305218680; forceNewPlan=false, retain=false 2023-03-08 19:55:12,442 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bf25ef5682cfde9aa90a5d6fe8229bdd, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33367,1678305218680; forceNewPlan=false, retain=false 2023-03-08 19:55:12,442 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d392b5a609f9661af565e4c73dab47f8, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33367,1678305218680; forceNewPlan=false, retain=false 2023-03-08 19:55:12,594 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=9c99479945efcf910b16288e607c83eb, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:12,594 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=bf25ef5682cfde9aa90a5d6fe8229bdd, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:12,594 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678305312594"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678305312594"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678305312594"}]},"ts":"1678305312594"} 2023-03-08 19:55:12,594 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=d392b5a609f9661af565e4c73dab47f8, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:12,594 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678305312594"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678305312594"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678305312594"}]},"ts":"1678305312594"} 2023-03-08 19:55:12,594 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678305312594"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678305312594"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678305312594"}]},"ts":"1678305312594"} 2023-03-08 19:55:12,596 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=81, state=RUNNABLE; OpenRegionProcedure 9c99479945efcf910b16288e607c83eb, server=jenkins-hbase4.apache.org,33367,1678305218680}] 2023-03-08 19:55:12,596 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=84, ppid=80, state=RUNNABLE; OpenRegionProcedure bf25ef5682cfde9aa90a5d6fe8229bdd, server=jenkins-hbase4.apache.org,33367,1678305218680}] 2023-03-08 19:55:12,598 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=85, ppid=82, state=RUNNABLE; OpenRegionProcedure d392b5a609f9661af565e4c73dab47f8, server=jenkins-hbase4.apache.org,33367,1678305218680}] 2023-03-08 19:55:12,708 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-08 19:55:12,752 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. 2023-03-08 19:55:12,752 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d392b5a609f9661af565e4c73dab47f8, NAME => 'testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8.', STARTKEY => '2', ENDKEY => ''} 2023-03-08 19:55:12,753 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink d392b5a609f9661af565e4c73dab47f8 2023-03-08 19:55:12,753 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:55:12,753 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d392b5a609f9661af565e4c73dab47f8 2023-03-08 19:55:12,753 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d392b5a609f9661af565e4c73dab47f8 2023-03-08 19:55:12,754 INFO [StoreOpener-d392b5a609f9661af565e4c73dab47f8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region d392b5a609f9661af565e4c73dab47f8 2023-03-08 19:55:12,756 DEBUG [StoreOpener-d392b5a609f9661af565e4c73dab47f8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1 2023-03-08 19:55:12,756 DEBUG [StoreOpener-d392b5a609f9661af565e4c73dab47f8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1 2023-03-08 19:55:12,756 INFO [StoreOpener-d392b5a609f9661af565e4c73dab47f8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d392b5a609f9661af565e4c73dab47f8 columnFamilyName family1 2023-03-08 19:55:12,757 INFO [StoreOpener-d392b5a609f9661af565e4c73dab47f8-1] regionserver.HStore(310): Store=d392b5a609f9661af565e4c73dab47f8/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:55:12,757 INFO [StoreOpener-d392b5a609f9661af565e4c73dab47f8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region d392b5a609f9661af565e4c73dab47f8 2023-03-08 19:55:12,758 DEBUG [StoreOpener-d392b5a609f9661af565e4c73dab47f8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2 2023-03-08 19:55:12,759 DEBUG [StoreOpener-d392b5a609f9661af565e4c73dab47f8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2 2023-03-08 19:55:12,759 INFO [StoreOpener-d392b5a609f9661af565e4c73dab47f8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d392b5a609f9661af565e4c73dab47f8 columnFamilyName family2 2023-03-08 19:55:12,760 INFO [StoreOpener-d392b5a609f9661af565e4c73dab47f8-1] regionserver.HStore(310): Store=d392b5a609f9661af565e4c73dab47f8/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:55:12,761 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8 2023-03-08 19:55:12,761 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8 2023-03-08 19:55:12,762 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 19:55:12,763 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d392b5a609f9661af565e4c73dab47f8 2023-03-08 19:55:12,765 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 19:55:12,766 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d392b5a609f9661af565e4c73dab47f8; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60059377, jitterRate=-0.10504554212093353}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 19:55:12,766 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d392b5a609f9661af565e4c73dab47f8: 2023-03-08 19:55:12,766 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8., pid=85, masterSystemTime=1678305312749 2023-03-08 19:55:12,767 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. 2023-03-08 19:55:12,768 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. 2023-03-08 19:55:12,768 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. 2023-03-08 19:55:12,768 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 9c99479945efcf910b16288e607c83eb, NAME => 'testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb.', STARTKEY => '1', ENDKEY => '2'} 2023-03-08 19:55:12,768 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=d392b5a609f9661af565e4c73dab47f8, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:12,768 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 9c99479945efcf910b16288e607c83eb 2023-03-08 19:55:12,768 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:55:12,768 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678305312768"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678305312768"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678305312768"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678305312768"}]},"ts":"1678305312768"} 2023-03-08 19:55:12,768 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 9c99479945efcf910b16288e607c83eb 2023-03-08 19:55:12,768 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 9c99479945efcf910b16288e607c83eb 2023-03-08 19:55:12,769 INFO [StoreOpener-9c99479945efcf910b16288e607c83eb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 9c99479945efcf910b16288e607c83eb 2023-03-08 19:55:12,770 DEBUG [StoreOpener-9c99479945efcf910b16288e607c83eb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1 2023-03-08 19:55:12,771 DEBUG [StoreOpener-9c99479945efcf910b16288e607c83eb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1 2023-03-08 19:55:12,771 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=85, resume processing ppid=82 2023-03-08 19:55:12,771 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=85, ppid=82, state=SUCCESS; OpenRegionProcedure d392b5a609f9661af565e4c73dab47f8, server=jenkins-hbase4.apache.org,33367,1678305218680 in 172 msec 2023-03-08 19:55:12,771 INFO [StoreOpener-9c99479945efcf910b16288e607c83eb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9c99479945efcf910b16288e607c83eb columnFamilyName family1 2023-03-08 19:55:12,772 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d392b5a609f9661af565e4c73dab47f8, ASSIGN in 331 msec 2023-03-08 19:55:12,772 INFO [StoreOpener-9c99479945efcf910b16288e607c83eb-1] regionserver.HStore(310): Store=9c99479945efcf910b16288e607c83eb/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:55:12,772 INFO [StoreOpener-9c99479945efcf910b16288e607c83eb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 9c99479945efcf910b16288e607c83eb 2023-03-08 19:55:12,773 DEBUG [StoreOpener-9c99479945efcf910b16288e607c83eb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2 2023-03-08 19:55:12,773 DEBUG [StoreOpener-9c99479945efcf910b16288e607c83eb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2 2023-03-08 19:55:12,774 INFO [StoreOpener-9c99479945efcf910b16288e607c83eb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9c99479945efcf910b16288e607c83eb columnFamilyName family2 2023-03-08 19:55:12,774 INFO [StoreOpener-9c99479945efcf910b16288e607c83eb-1] regionserver.HStore(310): Store=9c99479945efcf910b16288e607c83eb/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:55:12,775 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb 2023-03-08 19:55:12,775 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb 2023-03-08 19:55:12,776 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 19:55:12,777 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 9c99479945efcf910b16288e607c83eb 2023-03-08 19:55:12,779 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 19:55:12,779 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 9c99479945efcf910b16288e607c83eb; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61816163, jitterRate=-0.07886739075183868}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 19:55:12,779 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 9c99479945efcf910b16288e607c83eb: 2023-03-08 19:55:12,779 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb., pid=83, masterSystemTime=1678305312749 2023-03-08 19:55:12,781 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. 2023-03-08 19:55:12,781 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. 2023-03-08 19:55:12,781 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. 2023-03-08 19:55:12,781 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => bf25ef5682cfde9aa90a5d6fe8229bdd, NAME => 'testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd.', STARTKEY => '', ENDKEY => '1'} 2023-03-08 19:55:12,781 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=9c99479945efcf910b16288e607c83eb, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:12,781 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink bf25ef5682cfde9aa90a5d6fe8229bdd 2023-03-08 19:55:12,781 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678305312781"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678305312781"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678305312781"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678305312781"}]},"ts":"1678305312781"} 2023-03-08 19:55:12,781 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:55:12,781 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for bf25ef5682cfde9aa90a5d6fe8229bdd 2023-03-08 19:55:12,781 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for bf25ef5682cfde9aa90a5d6fe8229bdd 2023-03-08 19:55:12,782 INFO [StoreOpener-bf25ef5682cfde9aa90a5d6fe8229bdd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region bf25ef5682cfde9aa90a5d6fe8229bdd 2023-03-08 19:55:12,783 DEBUG [StoreOpener-bf25ef5682cfde9aa90a5d6fe8229bdd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1 2023-03-08 19:55:12,783 DEBUG [StoreOpener-bf25ef5682cfde9aa90a5d6fe8229bdd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1 2023-03-08 19:55:12,784 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=81 2023-03-08 19:55:12,784 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=81, state=SUCCESS; OpenRegionProcedure 9c99479945efcf910b16288e607c83eb, server=jenkins-hbase4.apache.org,33367,1678305218680 in 186 msec 2023-03-08 19:55:12,784 INFO [StoreOpener-bf25ef5682cfde9aa90a5d6fe8229bdd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bf25ef5682cfde9aa90a5d6fe8229bdd columnFamilyName family1 2023-03-08 19:55:12,785 INFO [StoreOpener-bf25ef5682cfde9aa90a5d6fe8229bdd-1] regionserver.HStore(310): Store=bf25ef5682cfde9aa90a5d6fe8229bdd/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:55:12,785 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=9c99479945efcf910b16288e607c83eb, ASSIGN in 345 msec 2023-03-08 19:55:12,785 INFO [StoreOpener-bf25ef5682cfde9aa90a5d6fe8229bdd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region bf25ef5682cfde9aa90a5d6fe8229bdd 2023-03-08 19:55:12,786 DEBUG [StoreOpener-bf25ef5682cfde9aa90a5d6fe8229bdd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2 2023-03-08 19:55:12,786 DEBUG [StoreOpener-bf25ef5682cfde9aa90a5d6fe8229bdd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2 2023-03-08 19:55:12,786 INFO [StoreOpener-bf25ef5682cfde9aa90a5d6fe8229bdd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bf25ef5682cfde9aa90a5d6fe8229bdd columnFamilyName family2 2023-03-08 19:55:12,787 INFO [StoreOpener-bf25ef5682cfde9aa90a5d6fe8229bdd-1] regionserver.HStore(310): Store=bf25ef5682cfde9aa90a5d6fe8229bdd/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:55:12,788 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd 2023-03-08 19:55:12,788 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd 2023-03-08 19:55:12,789 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 19:55:12,790 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for bf25ef5682cfde9aa90a5d6fe8229bdd 2023-03-08 19:55:12,791 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 19:55:12,792 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened bf25ef5682cfde9aa90a5d6fe8229bdd; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59714231, jitterRate=-0.11018861830234528}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 19:55:12,792 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for bf25ef5682cfde9aa90a5d6fe8229bdd: 2023-03-08 19:55:12,792 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd., pid=84, masterSystemTime=1678305312749 2023-03-08 19:55:12,794 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. 2023-03-08 19:55:12,794 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. 2023-03-08 19:55:12,794 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=bf25ef5682cfde9aa90a5d6fe8229bdd, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:12,794 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678305312794"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678305312794"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678305312794"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678305312794"}]},"ts":"1678305312794"} 2023-03-08 19:55:12,796 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=84, resume processing ppid=80 2023-03-08 19:55:12,796 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=84, ppid=80, state=SUCCESS; OpenRegionProcedure bf25ef5682cfde9aa90a5d6fe8229bdd, server=jenkins-hbase4.apache.org,33367,1678305218680 in 199 msec 2023-03-08 19:55:12,797 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=80, resume processing ppid=79 2023-03-08 19:55:12,798 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bf25ef5682cfde9aa90a5d6fe8229bdd, ASSIGN in 357 msec 2023-03-08 19:55:12,798 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 19:55:12,798 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678305312798"}]},"ts":"1678305312798"} 2023-03-08 19:55:12,799 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-03-08 19:55:12,801 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 19:55:12,802 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; CreateTableProcedure table=testCompactionWithHFileLink in 1.2000 sec 2023-03-08 19:55:13,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-08 19:55:13,709 INFO [Listener at localhost/34049] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testCompactionWithHFileLink, procId: 79 completed 2023-03-08 19:55:13,716 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 19:55:13,717 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 19:55:13,718 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 19:55:13,721 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:55:13,723 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-08 19:55:13,723 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-08 19:55:13,723 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:55:13,723 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-08 19:55:13,723 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-08 19:55:13,723 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:13,723 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 19:55:13,724 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:55:13,724 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:13,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:55:13,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:55:13,725 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:13,725 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:55:13,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 19:55:13,725 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:13,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:55:13,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 19:55:13,725 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-08 19:55:13,726 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-08 19:55:13,726 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-08 19:55:13,726 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:55:13,726 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-08 19:55:13,726 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. 2023-03-08 19:55:13,726 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. started... 2023-03-08 19:55:13,726 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. 2023-03-08 19:55:13,726 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. started... 2023-03-08 19:55:13,728 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 19:55:13,728 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:55:13,728 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing bf25ef5682cfde9aa90a5d6fe8229bdd 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-08 19:55:13,728 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. 2023-03-08 19:55:13,729 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. started... 2023-03-08 19:55:13,728 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9c99479945efcf910b16288e607c83eb 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-08 19:55:13,729 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing d392b5a609f9661af565e4c73dab47f8 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-08 19:55:14,146 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303088ee4da827de14300899601e052c73946 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303088ee4da827de14300899601e052c73946 2023-03-08 19:55:14,146 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308dd1c4643f5654221b89fb3ab8d9a8eb4 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308dd1c4643f5654221b89fb3ab8d9a8eb4 2023-03-08 19:55:14,149 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303088d68f70559bb4783aaa77c40b3a3e2c0 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303088d68f70559bb4783aaa77c40b3a3e2c0 2023-03-08 19:55:14,152 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/.tmp/family1/7372b15acde5432d9949bd6d62365bd8 2023-03-08 19:55:14,154 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/.tmp/family1/123a27e9c7024e02b0fbca054fd86280 2023-03-08 19:55:14,154 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/.tmp/family1/8cb2d3dd62cc42009c2420fabc400fb6 2023-03-08 19:55:14,177 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303085a588abaab334b20b603cfd602cb2f9e to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303085a588abaab334b20b603cfd602cb2f9e 2023-03-08 19:55:14,178 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303083e3cf9a6a99e4f1ca67c739bc1f51618 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303083e3cf9a6a99e4f1ca67c739bc1f51618 2023-03-08 19:55:14,578 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308f529b81cd56b4bc38d7251edc066f179 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308f529b81cd56b4bc38d7251edc066f179 2023-03-08 19:55:14,586 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/.tmp/family2/044f1e6d183045beae5b75645bb97f60 2023-03-08 19:55:14,586 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/.tmp/family2/07773024718d41078416d0bd2e154765 2023-03-08 19:55:14,590 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/.tmp/family1/8cb2d3dd62cc42009c2420fabc400fb6 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/8cb2d3dd62cc42009c2420fabc400fb6 2023-03-08 19:55:14,591 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/.tmp/family1/7372b15acde5432d9949bd6d62365bd8 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/7372b15acde5432d9949bd6d62365bd8 2023-03-08 19:55:14,594 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/8cb2d3dd62cc42009c2420fabc400fb6, entries=4, sequenceid=5, filesize=5.4 K 2023-03-08 19:55:14,594 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/.tmp/family2/044f1e6d183045beae5b75645bb97f60 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/044f1e6d183045beae5b75645bb97f60 2023-03-08 19:55:14,594 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/7372b15acde5432d9949bd6d62365bd8, entries=4, sequenceid=5, filesize=5.4 K 2023-03-08 19:55:14,595 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/.tmp/family2/07773024718d41078416d0bd2e154765 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/07773024718d41078416d0bd2e154765 2023-03-08 19:55:14,598 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/044f1e6d183045beae5b75645bb97f60, entries=2, sequenceid=5, filesize=5.1 K 2023-03-08 19:55:14,599 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for bf25ef5682cfde9aa90a5d6fe8229bdd in 871ms, sequenceid=5, compaction requested=false 2023-03-08 19:55:14,599 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-03-08 19:55:14,599 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/07773024718d41078416d0bd2e154765, entries=2, sequenceid=5, filesize=5.1 K 2023-03-08 19:55:14,599 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for bf25ef5682cfde9aa90a5d6fe8229bdd: 2023-03-08 19:55:14,599 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. 2023-03-08 19:55:14,599 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 19:55:14,600 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for d392b5a609f9661af565e4c73dab47f8 in 871ms, sequenceid=5, compaction requested=false 2023-03-08 19:55:14,600 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for d392b5a609f9661af565e4c73dab47f8: 2023-03-08 19:55:14,600 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. 2023-03-08 19:55:14,601 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 19:55:14,984 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/.tmp/family2/cdaeaeb4819a41cc88f69b9970a33dcb 2023-03-08 19:55:14,988 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/.tmp/family1/123a27e9c7024e02b0fbca054fd86280 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/123a27e9c7024e02b0fbca054fd86280 2023-03-08 19:55:14,991 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/123a27e9c7024e02b0fbca054fd86280, entries=4, sequenceid=5, filesize=5.4 K 2023-03-08 19:55:14,992 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/.tmp/family2/cdaeaeb4819a41cc88f69b9970a33dcb as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/cdaeaeb4819a41cc88f69b9970a33dcb 2023-03-08 19:55:14,995 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/cdaeaeb4819a41cc88f69b9970a33dcb, entries=2, sequenceid=5, filesize=5.1 K 2023-03-08 19:55:14,996 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 9c99479945efcf910b16288e607c83eb in 1268ms, sequenceid=5, compaction requested=false 2023-03-08 19:55:14,996 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9c99479945efcf910b16288e607c83eb: 2023-03-08 19:55:14,996 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. 2023-03-08 19:55:14,996 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 19:55:14,996 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 19:55:14,996 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:14,996 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-08 19:55:14,996 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-08 19:55:14,998 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:14,998 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 19:55:14,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:14,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:14,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:14,999 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 19:55:14,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:14,999 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:55:14,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:14,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 19:55:14,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:14,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:15,000 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-08 19:55:15,000 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@c4f381f[Count = 0] remaining members to acquire global barrier 2023-03-08 19:55:15,000 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-08 19:55:15,000 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 19:55:15,002 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 19:55:15,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 19:55:15,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 19:55:15,002 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-08 19:55:15,002 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:15,002 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:55:15,002 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-08 19:55:15,002 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:55:15,004 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:15,004 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-08 19:55:15,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:15,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:15,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:15,004 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:55:15,004 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-08 19:55:15,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:15,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:15,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 19:55:15,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:15,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:15,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 19:55:15,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:15,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:55:15,007 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-08 19:55:15,007 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-08 19:55:15,007 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:55:15,007 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:55:15,007 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-08 19:55:15,007 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:55:15,009 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:15,009 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:15,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:15,009 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:55:15,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:15,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:15,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:15,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:15,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:55:15,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:55:15,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:15,009 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:15,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:15,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 19:55:15,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:15,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 19:55:15,011 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:15,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:15,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:15,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 19:55:15,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:15,013 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:15,013 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:55:15,013 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 19:55:15,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:55:15,013 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-08 19:55:15,013 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:55:15,013 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 19:55:15,013 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:55:15,013 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:55:15,013 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:15,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:55:15,013 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 19:55:15,013 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 19:55:15,013 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:15,013 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-08 19:55:15,013 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:55:15,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:55:15,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:55:15,114 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:55:15,115 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:55:15,116 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 19:55:15,116 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 19:55:15,116 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 19:55:15,118 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:55:15,120 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-08 19:55:15,120 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-08 19:55:15,121 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-08 19:55:15,121 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:55:15,121 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-08 19:55:15,121 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-08 19:55:15,121 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:15,121 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 19:55:15,124 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:55:15,124 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:15,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:55:15,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:55:15,124 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:15,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 19:55:15,124 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:55:15,125 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:15,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:55:15,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 19:55:15,125 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-08 19:55:15,125 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-03-08 19:55:15,125 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-08 19:55:15,125 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-08 19:55:15,125 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:55:15,125 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-08 19:55:15,126 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. 2023-03-08 19:55:15,126 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. started... 2023-03-08 19:55:15,126 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. 2023-03-08 19:55:15,126 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. started... 2023-03-08 19:55:15,126 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9c99479945efcf910b16288e607c83eb 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-08 19:55:15,126 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing bf25ef5682cfde9aa90a5d6fe8229bdd 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-08 19:55:15,128 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 19:55:15,128 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:55:15,128 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. 2023-03-08 19:55:15,128 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. started... 2023-03-08 19:55:15,128 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing d392b5a609f9661af565e4c73dab47f8 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-08 19:55:15,554 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308e7a0a9d515b24f63a4013e970fb178a0 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308e7a0a9d515b24f63a4013e970fb178a0 2023-03-08 19:55:15,554 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303088ffe3f5d37194c0fb226cac523d21fef to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303088ffe3f5d37194c0fb226cac523d21fef 2023-03-08 19:55:15,558 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303089f52416d158d4f62ac1989a0c5c86166 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303089f52416d158d4f62ac1989a0c5c86166 2023-03-08 19:55:15,560 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/.tmp/family1/e39643d9b2424962b7f3883e1e919f9a 2023-03-08 19:55:15,564 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/.tmp/family1/a433c1e1bdb446f385d1156aa8209307 2023-03-08 19:55:15,970 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/.tmp/family1/af1975c5b195440797e41fe2eba32cc4 2023-03-08 19:55:15,982 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308b5e613f2acc74fef836cd60172ea0d37 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230308b5e613f2acc74fef836cd60172ea0d37 2023-03-08 19:55:15,982 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308b47202b6a5174e87a5ae0040bc3efca7 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308b47202b6a5174e87a5ae0040bc3efca7 2023-03-08 19:55:15,987 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308a8d8d630c1774bd8bbc29cb46a189584 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308a8d8d630c1774bd8bbc29cb46a189584 2023-03-08 19:55:15,994 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/.tmp/family2/3e46bffcb555430ca182ded732797e54 2023-03-08 19:55:15,995 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/.tmp/family2/d9b3dfdcafcf4aa3b87dfdc6c038bf9e 2023-03-08 19:55:15,998 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/.tmp/family1/e39643d9b2424962b7f3883e1e919f9a as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/e39643d9b2424962b7f3883e1e919f9a 2023-03-08 19:55:15,999 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/.tmp/family1/af1975c5b195440797e41fe2eba32cc4 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/af1975c5b195440797e41fe2eba32cc4 2023-03-08 19:55:16,001 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/e39643d9b2424962b7f3883e1e919f9a, entries=4, sequenceid=9, filesize=5.4 K 2023-03-08 19:55:16,002 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/.tmp/family2/3e46bffcb555430ca182ded732797e54 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/3e46bffcb555430ca182ded732797e54 2023-03-08 19:55:16,003 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/af1975c5b195440797e41fe2eba32cc4, entries=4, sequenceid=9, filesize=5.4 K 2023-03-08 19:55:16,004 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/.tmp/family2/d9b3dfdcafcf4aa3b87dfdc6c038bf9e as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/d9b3dfdcafcf4aa3b87dfdc6c038bf9e 2023-03-08 19:55:16,006 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/3e46bffcb555430ca182ded732797e54, entries=2, sequenceid=9, filesize=5.1 K 2023-03-08 19:55:16,007 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for d392b5a609f9661af565e4c73dab47f8 in 879ms, sequenceid=9, compaction requested=false 2023-03-08 19:55:16,007 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for d392b5a609f9661af565e4c73dab47f8: 2023-03-08 19:55:16,007 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. 2023-03-08 19:55:16,007 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 19:55:16,008 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/d9b3dfdcafcf4aa3b87dfdc6c038bf9e, entries=2, sequenceid=9, filesize=5.1 K 2023-03-08 19:55:16,009 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for bf25ef5682cfde9aa90a5d6fe8229bdd in 883ms, sequenceid=9, compaction requested=false 2023-03-08 19:55:16,009 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for bf25ef5682cfde9aa90a5d6fe8229bdd: 2023-03-08 19:55:16,009 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. 2023-03-08 19:55:16,009 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 19:55:16,388 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/.tmp/family2/68353db2020f4adeba9aeb341538de89 2023-03-08 19:55:16,393 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/.tmp/family1/a433c1e1bdb446f385d1156aa8209307 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/a433c1e1bdb446f385d1156aa8209307 2023-03-08 19:55:16,396 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/a433c1e1bdb446f385d1156aa8209307, entries=4, sequenceid=9, filesize=5.4 K 2023-03-08 19:55:16,397 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/.tmp/family2/68353db2020f4adeba9aeb341538de89 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/68353db2020f4adeba9aeb341538de89 2023-03-08 19:55:16,400 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/68353db2020f4adeba9aeb341538de89, entries=2, sequenceid=9, filesize=5.1 K 2023-03-08 19:55:16,401 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 9c99479945efcf910b16288e607c83eb in 1275ms, sequenceid=9, compaction requested=false 2023-03-08 19:55:16,401 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9c99479945efcf910b16288e607c83eb: 2023-03-08 19:55:16,401 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. 2023-03-08 19:55:16,401 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 19:55:16,401 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 19:55:16,401 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:16,402 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-08 19:55:16,402 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-08 19:55:16,403 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 19:55:16,403 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:16,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:16,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:16,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:16,404 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 19:55:16,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:16,404 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:55:16,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:16,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 19:55:16,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:16,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:16,405 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-08 19:55:16,405 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6f698a75[Count = 0] remaining members to acquire global barrier 2023-03-08 19:55:16,405 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-08 19:55:16,405 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 19:55:16,406 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 19:55:16,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 19:55:16,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 19:55:16,406 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-08 19:55:16,406 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-08 19:55:16,406 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:55:16,406 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:16,406 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:55:16,407 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-08 19:55:16,407 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:16,407 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:55:16,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:16,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:16,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:16,408 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-08 19:55:16,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:16,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:16,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 19:55:16,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:16,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:16,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 19:55:16,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:16,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:55:16,410 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-08 19:55:16,410 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-08 19:55:16,410 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:55:16,410 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:55:16,410 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-08 19:55:16,410 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:55:16,412 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:16,412 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:16,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:16,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:16,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:16,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:16,412 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:55:16,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:16,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:16,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:55:16,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:55:16,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 19:55:16,413 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:16,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:16,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:16,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 19:55:16,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:16,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:16,414 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:16,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 19:55:16,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:16,416 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:55:16,416 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:16,416 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:55:16,416 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 19:55:16,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:55:16,416 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:55:16,416 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-08 19:55:16,416 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:55:16,416 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 19:55:16,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:55:16,416 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:16,417 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 19:55:16,417 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 19:55:16,417 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:16,417 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-08 19:55:16,417 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:55:16,417 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:55:16,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:55:16,517 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:55:16,518 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:55:16,519 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 19:55:16,519 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 19:55:16,519 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 19:55:16,522 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:55:16,524 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-08 19:55:16,524 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-08 19:55:16,524 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-08 19:55:16,524 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:55:16,524 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-08 19:55:16,524 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-08 19:55:16,524 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:16,524 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 19:55:16,526 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:16,526 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:55:16,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:55:16,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:55:16,526 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:16,526 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:55:16,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 19:55:16,527 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:16,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:55:16,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 19:55:16,527 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-08 19:55:16,527 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-03-08 19:55:16,527 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-08 19:55:16,527 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-08 19:55:16,528 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:55:16,528 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-08 19:55:16,528 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. 2023-03-08 19:55:16,528 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. started... 2023-03-08 19:55:16,528 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9c99479945efcf910b16288e607c83eb 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-08 19:55:16,528 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 19:55:16,528 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:55:16,528 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. 2023-03-08 19:55:16,528 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. 2023-03-08 19:55:16,528 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. started... 2023-03-08 19:55:16,528 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. started... 2023-03-08 19:55:16,528 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing bf25ef5682cfde9aa90a5d6fe8229bdd 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-08 19:55:16,528 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing d392b5a609f9661af565e4c73dab47f8 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-08 19:55:16,548 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303083653e76c42154508bdba69de9d712f83 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303083653e76c42154508bdba69de9d712f83 2023-03-08 19:55:16,824 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 19:55:16,942 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023030803a51c78c3a9414481e4489a8fc504ad to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030803a51c78c3a9414481e4489a8fc504ad 2023-03-08 19:55:16,947 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/.tmp/family1/c285d649295f485d986c468697edc0ce 2023-03-08 19:55:16,948 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308b87a29e85e7140f29524a1dd29216255 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308b87a29e85e7140f29524a1dd29216255 2023-03-08 19:55:16,953 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/.tmp/family1/9a28a7cd1d794cde8d25ec94745b3bb9 2023-03-08 19:55:16,953 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/.tmp/family1/046ec2868b184c3e84b6105bf189d3bd 2023-03-08 19:55:17,364 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308530634480de84dbdbb673bdb4d36dcfe to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230308530634480de84dbdbb673bdb4d36dcfe 2023-03-08 19:55:17,369 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308063437f66d8b4c478dc3e48cccd84c31 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308063437f66d8b4c478dc3e48cccd84c31 2023-03-08 19:55:17,369 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/.tmp/family2/6eeb48f439a34bc9ae7493396a8ad303 2023-03-08 19:55:17,372 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023030843a51fd1ba354014a3638289006916d9 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023030843a51fd1ba354014a3638289006916d9 2023-03-08 19:55:17,375 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/.tmp/family1/c285d649295f485d986c468697edc0ce as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/c285d649295f485d986c468697edc0ce 2023-03-08 19:55:17,379 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/c285d649295f485d986c468697edc0ce, entries=4, sequenceid=13, filesize=5.4 K 2023-03-08 19:55:17,379 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/.tmp/family2/6eeb48f439a34bc9ae7493396a8ad303 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/6eeb48f439a34bc9ae7493396a8ad303 2023-03-08 19:55:17,383 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/6eeb48f439a34bc9ae7493396a8ad303, entries=2, sequenceid=13, filesize=5.1 K 2023-03-08 19:55:17,383 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for d392b5a609f9661af565e4c73dab47f8 in 855ms, sequenceid=13, compaction requested=true 2023-03-08 19:55:17,383 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for d392b5a609f9661af565e4c73dab47f8: 2023-03-08 19:55:17,383 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. 2023-03-08 19:55:17,384 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 19:55:17,659 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(152): Removing adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-03-08 19:55:17,776 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/.tmp/family2/7957d21df4a84e90b22385e13d00adde 2023-03-08 19:55:17,777 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/.tmp/family2/d97fbd02a2e04d4cb97a3bfe7772fb6e 2023-03-08 19:55:17,781 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/.tmp/family1/9a28a7cd1d794cde8d25ec94745b3bb9 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/9a28a7cd1d794cde8d25ec94745b3bb9 2023-03-08 19:55:17,782 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/.tmp/family1/046ec2868b184c3e84b6105bf189d3bd as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/046ec2868b184c3e84b6105bf189d3bd 2023-03-08 19:55:17,786 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/9a28a7cd1d794cde8d25ec94745b3bb9, entries=4, sequenceid=13, filesize=5.4 K 2023-03-08 19:55:17,787 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/046ec2868b184c3e84b6105bf189d3bd, entries=4, sequenceid=13, filesize=5.4 K 2023-03-08 19:55:17,788 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/.tmp/family2/7957d21df4a84e90b22385e13d00adde as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/7957d21df4a84e90b22385e13d00adde 2023-03-08 19:55:17,788 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/.tmp/family2/d97fbd02a2e04d4cb97a3bfe7772fb6e as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/d97fbd02a2e04d4cb97a3bfe7772fb6e 2023-03-08 19:55:17,801 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/7957d21df4a84e90b22385e13d00adde, entries=2, sequenceid=13, filesize=5.1 K 2023-03-08 19:55:17,801 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/d97fbd02a2e04d4cb97a3bfe7772fb6e, entries=2, sequenceid=13, filesize=5.1 K 2023-03-08 19:55:17,802 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for bf25ef5682cfde9aa90a5d6fe8229bdd in 1274ms, sequenceid=13, compaction requested=true 2023-03-08 19:55:17,802 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 9c99479945efcf910b16288e607c83eb in 1274ms, sequenceid=13, compaction requested=true 2023-03-08 19:55:17,802 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for bf25ef5682cfde9aa90a5d6fe8229bdd: 2023-03-08 19:55:17,802 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9c99479945efcf910b16288e607c83eb: 2023-03-08 19:55:17,802 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. 2023-03-08 19:55:17,802 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. 2023-03-08 19:55:17,802 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 19:55:17,802 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 19:55:17,802 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 19:55:17,802 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:17,802 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-08 19:55:17,803 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-08 19:55:17,804 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:17,804 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 19:55:17,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:17,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:17,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:17,805 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 19:55:17,805 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:55:17,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:17,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:17,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 19:55:17,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:17,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:17,806 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-08 19:55:17,806 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@20da2dfc[Count = 0] remaining members to acquire global barrier 2023-03-08 19:55:17,806 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-08 19:55:17,806 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 19:55:17,808 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 19:55:17,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 19:55:17,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 19:55:17,809 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:17,809 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:55:17,809 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-08 19:55:17,809 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-08 19:55:17,809 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:55:17,810 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:17,810 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-08 19:55:17,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:17,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:17,810 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:55:17,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:17,810 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-08 19:55:17,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:17,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:17,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 19:55:17,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:17,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:17,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 19:55:17,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:17,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:55:17,812 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-08 19:55:17,812 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-08 19:55:17,812 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:55:17,812 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:55:17,812 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-08 19:55:17,813 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:55:17,816 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:17,816 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:17,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:17,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:17,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:17,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:17,816 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:55:17,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:17,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:17,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:55:17,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:55:17,817 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:17,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 19:55:17,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:17,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:17,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 19:55:17,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:17,818 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:17,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:17,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 19:55:17,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:17,820 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:17,820 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 19:55:17,820 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:55:17,820 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:55:17,820 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-08 19:55:17,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:55:17,820 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 19:55:17,820 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-08 19:55:17,821 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:55:17,820 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:17,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:55:17,820 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:55:17,821 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 19:55:17,821 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 19:55:17,821 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:17,821 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-08 19:55:17,821 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:55:17,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:55:17,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:55:17,921 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:55:17,922 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:55:17,923 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 19:55:17,923 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 19:55:17,923 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 19:55:17,926 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:55:17,928 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-08 19:55:17,928 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-08 19:55:17,928 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-08 19:55:17,928 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:55:17,928 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-08 19:55:17,928 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-08 19:55:17,929 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:17,929 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 19:55:17,930 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:17,930 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:55:17,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:55:17,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:55:17,930 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:17,930 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:55:17,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 19:55:17,931 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:17,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:55:17,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 19:55:17,931 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-08 19:55:17,931 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-03-08 19:55:17,931 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-08 19:55:17,931 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-08 19:55:17,931 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:55:17,931 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-08 19:55:17,933 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. 2023-03-08 19:55:17,933 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. started... 2023-03-08 19:55:17,933 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. 2023-03-08 19:55:17,933 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9c99479945efcf910b16288e607c83eb 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-08 19:55:17,933 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. started... 2023-03-08 19:55:17,933 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing bf25ef5682cfde9aa90a5d6fe8229bdd 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-08 19:55:17,933 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 19:55:17,933 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:55:17,934 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. 2023-03-08 19:55:17,934 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. started... 2023-03-08 19:55:17,934 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing d392b5a609f9661af565e4c73dab47f8 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-08 19:55:17,958 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308c0451f6624404cb9a261ce5d697fda74 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308c0451f6624404cb9a261ce5d697fda74 2023-03-08 19:55:18,357 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308d3c2ea31978640cf9022c0e5fb535d1c to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308d3c2ea31978640cf9022c0e5fb535d1c 2023-03-08 19:55:18,358 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308c5433010db254bcfb902142789f105cc to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308c5433010db254bcfb902142789f105cc 2023-03-08 19:55:18,362 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/.tmp/family1/c9ebe20caf5d4c2cb57adb8d0820d554 2023-03-08 19:55:18,762 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/.tmp/family1/f7d2cb4000d943219c0ae3c05e060127 2023-03-08 19:55:18,762 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/.tmp/family1/67ebbe2e123e43d18dd53054062690ec 2023-03-08 19:55:18,776 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308336c8ea8d5a648e5a58dd344250b0346 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308336c8ea8d5a648e5a58dd344250b0346 2023-03-08 19:55:18,776 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303085c368878706a4756a425c85247df761f to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303085c368878706a4756a425c85247df761f 2023-03-08 19:55:18,786 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/.tmp/family2/a84f2cc20b89462cab9556cca240ba20 2023-03-08 19:55:18,790 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/.tmp/family1/67ebbe2e123e43d18dd53054062690ec as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/67ebbe2e123e43d18dd53054062690ec 2023-03-08 19:55:18,794 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/67ebbe2e123e43d18dd53054062690ec, entries=4, sequenceid=17, filesize=5.4 K 2023-03-08 19:55:18,795 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/.tmp/family2/a84f2cc20b89462cab9556cca240ba20 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/a84f2cc20b89462cab9556cca240ba20 2023-03-08 19:55:18,799 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/a84f2cc20b89462cab9556cca240ba20, entries=2, sequenceid=17, filesize=5.1 K 2023-03-08 19:55:18,800 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for d392b5a609f9661af565e4c73dab47f8 in 866ms, sequenceid=17, compaction requested=true 2023-03-08 19:55:18,800 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for d392b5a609f9661af565e4c73dab47f8: 2023-03-08 19:55:18,800 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. 2023-03-08 19:55:18,800 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 19:55:19,182 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023030876751d04d8044b56831dbe61c59ec030 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023030876751d04d8044b56831dbe61c59ec030 2023-03-08 19:55:19,186 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/.tmp/family2/e323979e94844e49a28882288ab3a208 2023-03-08 19:55:19,189 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/.tmp/family1/c9ebe20caf5d4c2cb57adb8d0820d554 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/c9ebe20caf5d4c2cb57adb8d0820d554 2023-03-08 19:55:19,193 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/c9ebe20caf5d4c2cb57adb8d0820d554, entries=4, sequenceid=17, filesize=5.4 K 2023-03-08 19:55:19,194 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/.tmp/family2/e323979e94844e49a28882288ab3a208 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/e323979e94844e49a28882288ab3a208 2023-03-08 19:55:19,198 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/e323979e94844e49a28882288ab3a208, entries=2, sequenceid=17, filesize=5.1 K 2023-03-08 19:55:19,198 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for bf25ef5682cfde9aa90a5d6fe8229bdd in 1265ms, sequenceid=17, compaction requested=true 2023-03-08 19:55:19,198 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for bf25ef5682cfde9aa90a5d6fe8229bdd: 2023-03-08 19:55:19,198 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. 2023-03-08 19:55:19,199 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 19:55:19,587 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/.tmp/family2/3f39c3b5bff246788da3275e205043fb 2023-03-08 19:55:19,591 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/.tmp/family1/f7d2cb4000d943219c0ae3c05e060127 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/f7d2cb4000d943219c0ae3c05e060127 2023-03-08 19:55:19,595 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/f7d2cb4000d943219c0ae3c05e060127, entries=4, sequenceid=17, filesize=5.4 K 2023-03-08 19:55:19,596 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/.tmp/family2/3f39c3b5bff246788da3275e205043fb as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/3f39c3b5bff246788da3275e205043fb 2023-03-08 19:55:19,599 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/3f39c3b5bff246788da3275e205043fb, entries=2, sequenceid=17, filesize=5.1 K 2023-03-08 19:55:19,600 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 9c99479945efcf910b16288e607c83eb in 1667ms, sequenceid=17, compaction requested=true 2023-03-08 19:55:19,600 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9c99479945efcf910b16288e607c83eb: 2023-03-08 19:55:19,600 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. 2023-03-08 19:55:19,600 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 19:55:19,600 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 19:55:19,600 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:19,600 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-08 19:55:19,600 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-08 19:55:19,602 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:19,602 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 19:55:19,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:19,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:19,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:19,603 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 19:55:19,603 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:55:19,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:19,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:19,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 19:55:19,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:19,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:19,604 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-08 19:55:19,604 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@e981d0e[Count = 0] remaining members to acquire global barrier 2023-03-08 19:55:19,604 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-08 19:55:19,604 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 19:55:19,605 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 19:55:19,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 19:55:19,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 19:55:19,605 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-08 19:55:19,605 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-08 19:55:19,605 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:55:19,605 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:19,605 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:55:19,607 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:19,607 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-08 19:55:19,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:19,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:19,607 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:55:19,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:19,607 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-08 19:55:19,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:19,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:19,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 19:55:19,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:19,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:19,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 19:55:19,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:19,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:55:19,609 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-08 19:55:19,609 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-08 19:55:19,609 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:55:19,609 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:55:19,609 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-08 19:55:19,609 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:55:19,610 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:19,610 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:19,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:19,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:19,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:19,611 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:55:19,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:19,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:19,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:19,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:55:19,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:55:19,611 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:19,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 19:55:19,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:19,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:19,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 19:55:19,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:19,612 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:19,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:19,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 19:55:19,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:19,615 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:19,616 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:55:19,616 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 19:55:19,616 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:55:19,616 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:55:19,616 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 19:55:19,616 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-08 19:55:19,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:55:19,616 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:19,616 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-08 19:55:19,616 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:55:19,616 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 19:55:19,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:55:19,616 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 19:55:19,616 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:19,616 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-08 19:55:19,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:55:19,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:55:19,616 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:55:19,716 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:55:19,717 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:55:19,727 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1580): Client=jenkins//172.31.14.131 snapshot request for:{ ss=snaptb-1678305319718 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-08 19:55:19,728 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1678305319728 (current time:1678305319728). 2023-03-08 19:55:19,728 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-03-08 19:55:19,728 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] zookeeper.ReadOnlyZKClient(139): Connect 0x54d61a12 to 127.0.0.1:56751 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 19:55:19,734 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6dbbe600, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 19:55:19,738 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 19:55:19,742 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53774, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 19:55:19,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x54d61a12 to 127.0.0.1:56751 2023-03-08 19:55:19,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 19:55:19,743 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-03-08 19:55:19,744 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] snapshot.SnapshotManager(648): Table enabled, starting distributed snapshots for { ss=snaptb-1678305319718 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-08 19:55:19,754 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure2.ProcedureExecutor(1029): Stored pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-08 19:55:19,755 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-08 19:55:19,756 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=86, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1678305919756 2023-03-08 19:55:19,756 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] snapshot.SnapshotManager(653): Started snapshot: { ss=snaptb-1678305319718 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-08 19:55:19,756 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running FLUSH table snapshot snaptb-1678305319718 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-03-08 19:55:19,758 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=86, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-08 19:55:19,758 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure2.ProcedureExecutor(1029): Stored pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-03-08 19:55:19,759 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=86, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 5 msec 2023-03-08 19:55:19,760 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-03-08 19:55:19,761 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=87, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED; timeout=600000, timestamp=1678305919761 2023-03-08 19:55:19,762 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=snaptb-1678305319718 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-03-08 19:55:19,762 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-03-08 19:55:19,863 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 19:55:19,865 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678305319718 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 19:55:19,867 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1678305319718 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-08 19:55:19,868 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-03-08 19:55:20,068 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 19:55:20,069 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678305319718 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 19:55:20,069 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1678305319718 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-08 19:55:20,069 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-03-08 19:55:20,168 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure.ProcedureCoordinator(165): Submitting procedure snaptb-1678305319718 2023-03-08 19:55:20,179 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'snaptb-1678305319718' 2023-03-08 19:55:20,179 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-03-08 19:55:20,179 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'snaptb-1678305319718' starting 'acquire' 2023-03-08 19:55:20,179 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'snaptb-1678305319718', kicking off acquire phase on members. 2023-03-08 19:55:20,179 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1678305319718 2023-03-08 19:55:20,179 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/online-snapshot/acquired/snaptb-1678305319718 2023-03-08 19:55:20,181 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-03-08 19:55:20,181 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/online-snapshot/acquired/snaptb-1678305319718/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:20,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-03-08 19:55:20,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-03-08 19:55:20,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/snaptb-1678305319718 2023-03-08 19:55:20,181 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/acquired/snaptb-1678305319718/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:20,181 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:55:20,182 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1678305319718 2023-03-08 19:55:20,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 68 2023-03-08 19:55:20,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/snaptb-1678305319718 2023-03-08 19:55:20,182 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot snaptb-1678305319718 from table testCompactionWithHFileLink type FLUSH 2023-03-08 19:55:20,184 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:snaptb-1678305319718 2023-03-08 19:55:20,188 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'snaptb-1678305319718' with timeout 300000ms 2023-03-08 19:55:20,188 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-03-08 19:55:20,188 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'snaptb-1678305319718' starting 'acquire' stage 2023-03-08 19:55:20,188 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'snaptb-1678305319718' locally acquired 2023-03-08 19:55:20,188 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (snaptb-1678305319718) in zk 2023-03-08 19:55:20,191 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1678305319718/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:20,191 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/snaptb-1678305319718 2023-03-08 19:55:20,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/acquired/snaptb-1678305319718/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:20,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:20,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-03-08 19:55:20,191 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1678305319718 2023-03-08 19:55:20,191 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'snaptb-1678305319718' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:55:20,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:20,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:20,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678305319718 2023-03-08 19:55:20,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:20,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:20,193 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'snaptb-1678305319718' on coordinator 2023-03-08 19:55:20,193 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7b624302[Count = 0] remaining members to acquire global barrier 2023-03-08 19:55:20,193 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'snaptb-1678305319718' starting 'in-barrier' execution. 2023-03-08 19:55:20,193 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/online-snapshot/reached/snaptb-1678305319718 2023-03-08 19:55:20,194 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1678305319718 2023-03-08 19:55:20,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/snaptb-1678305319718 2023-03-08 19:55:20,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/snaptb-1678305319718 2023-03-08 19:55:20,194 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'snaptb-1678305319718' received 'reached' from coordinator. 2023-03-08 19:55:20,194 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1678305319718/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:20,195 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:55:20,195 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. 2023-03-08 19:55:20,195 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. 2023-03-08 19:55:20,195 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. 2023-03-08 19:55:20,195 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] snapshot.FlushSnapshotSubprocedure(170): Flush Snapshot Tasks submitted for 3 regions 2023-03-08 19:55:20,196 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(301): Waiting for local region snapshots to finish. 2023-03-08 19:55:20,196 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. started... 2023-03-08 19:55:20,196 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. started... 2023-03-08 19:55:20,196 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. started... 2023-03-08 19:55:20,196 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-2] regionserver.HRegion(2446): Flush status journal for d392b5a609f9661af565e4c73dab47f8: 2023-03-08 19:55:20,196 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-0] regionserver.HRegion(2446): Flush status journal for 9c99479945efcf910b16288e607c83eb: 2023-03-08 19:55:20,196 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-1] regionserver.HRegion(2446): Flush status journal for bf25ef5682cfde9aa90a5d6fe8229bdd: 2023-03-08 19:55:20,197 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-2] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8.' region-info for snapshot=snaptb-1678305319718 2023-03-08 19:55:20,197 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-1] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd.' region-info for snapshot=snaptb-1678305319718 2023-03-08 19:55:20,197 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-0] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb.' region-info for snapshot=snaptb-1678305319718 2023-03-08 19:55:20,202 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-1] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-03-08 19:55:20,202 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-2] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-03-08 19:55:20,202 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-0] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-03-08 19:55:20,207 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/7372b15acde5432d9949bd6d62365bd8, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/e39643d9b2424962b7f3883e1e919f9a, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/c285d649295f485d986c468697edc0ce, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/67ebbe2e123e43d18dd53054062690ec] hfiles 2023-03-08 19:55:20,207 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/8cb2d3dd62cc42009c2420fabc400fb6, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/af1975c5b195440797e41fe2eba32cc4, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/046ec2868b184c3e84b6105bf189d3bd, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/c9ebe20caf5d4c2cb57adb8d0820d554] hfiles 2023-03-08 19:55:20,207 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/123a27e9c7024e02b0fbca054fd86280, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/a433c1e1bdb446f385d1156aa8209307, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/9a28a7cd1d794cde8d25ec94745b3bb9, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/f7d2cb4000d943219c0ae3c05e060127] hfiles 2023-03-08 19:55:20,207 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/8cb2d3dd62cc42009c2420fabc400fb6 for snapshot=snaptb-1678305319718 2023-03-08 19:55:20,207 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/7372b15acde5432d9949bd6d62365bd8 for snapshot=snaptb-1678305319718 2023-03-08 19:55:20,207 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/123a27e9c7024e02b0fbca054fd86280 for snapshot=snaptb-1678305319718 2023-03-08 19:55:20,212 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/e39643d9b2424962b7f3883e1e919f9a for snapshot=snaptb-1678305319718 2023-03-08 19:55:20,212 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/af1975c5b195440797e41fe2eba32cc4 for snapshot=snaptb-1678305319718 2023-03-08 19:55:20,212 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/a433c1e1bdb446f385d1156aa8209307 for snapshot=snaptb-1678305319718 2023-03-08 19:55:20,212 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/046ec2868b184c3e84b6105bf189d3bd for snapshot=snaptb-1678305319718 2023-03-08 19:55:20,212 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/c285d649295f485d986c468697edc0ce for snapshot=snaptb-1678305319718 2023-03-08 19:55:20,212 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/c9ebe20caf5d4c2cb57adb8d0820d554 for snapshot=snaptb-1678305319718 2023-03-08 19:55:20,212 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/9a28a7cd1d794cde8d25ec94745b3bb9 for snapshot=snaptb-1678305319718 2023-03-08 19:55:20,212 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/67ebbe2e123e43d18dd53054062690ec for snapshot=snaptb-1678305319718 2023-03-08 19:55:20,212 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/f7d2cb4000d943219c0ae3c05e060127 for snapshot=snaptb-1678305319718 2023-03-08 19:55:20,212 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/07773024718d41078416d0bd2e154765, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/3e46bffcb555430ca182ded732797e54, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/6eeb48f439a34bc9ae7493396a8ad303, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/a84f2cc20b89462cab9556cca240ba20] hfiles 2023-03-08 19:55:20,212 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/cdaeaeb4819a41cc88f69b9970a33dcb, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/68353db2020f4adeba9aeb341538de89, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/7957d21df4a84e90b22385e13d00adde, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/3f39c3b5bff246788da3275e205043fb] hfiles 2023-03-08 19:55:20,212 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/07773024718d41078416d0bd2e154765 for snapshot=snaptb-1678305319718 2023-03-08 19:55:20,212 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/3e46bffcb555430ca182ded732797e54 for snapshot=snaptb-1678305319718 2023-03-08 19:55:20,212 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/044f1e6d183045beae5b75645bb97f60, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/d9b3dfdcafcf4aa3b87dfdc6c038bf9e, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/d97fbd02a2e04d4cb97a3bfe7772fb6e, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/e323979e94844e49a28882288ab3a208] hfiles 2023-03-08 19:55:20,212 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/6eeb48f439a34bc9ae7493396a8ad303 for snapshot=snaptb-1678305319718 2023-03-08 19:55:20,212 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/cdaeaeb4819a41cc88f69b9970a33dcb for snapshot=snaptb-1678305319718 2023-03-08 19:55:20,212 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/a84f2cc20b89462cab9556cca240ba20 for snapshot=snaptb-1678305319718 2023-03-08 19:55:20,212 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/044f1e6d183045beae5b75645bb97f60 for snapshot=snaptb-1678305319718 2023-03-08 19:55:20,212 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/68353db2020f4adeba9aeb341538de89 for snapshot=snaptb-1678305319718 2023-03-08 19:55:20,213 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/7957d21df4a84e90b22385e13d00adde for snapshot=snaptb-1678305319718 2023-03-08 19:55:20,213 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/d9b3dfdcafcf4aa3b87dfdc6c038bf9e for snapshot=snaptb-1678305319718 2023-03-08 19:55:20,213 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/3f39c3b5bff246788da3275e205043fb for snapshot=snaptb-1678305319718 2023-03-08 19:55:20,213 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/d97fbd02a2e04d4cb97a3bfe7772fb6e for snapshot=snaptb-1678305319718 2023-03-08 19:55:20,213 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/e323979e94844e49a28882288ab3a208 for snapshot=snaptb-1678305319718 2023-03-08 19:55:20,370 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 19:55:20,370 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678305319718 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 19:55:20,371 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1678305319718 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-08 19:55:20,371 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-03-08 19:55:20,622 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. completed. 2023-03-08 19:55:20,622 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. 2023-03-08 19:55:20,622 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. completed. 2023-03-08 19:55:20,622 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. 2023-03-08 19:55:20,622 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 1/3 local region snapshots. 2023-03-08 19:55:20,622 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 2/3 local region snapshots. 2023-03-08 19:55:20,629 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. completed. 2023-03-08 19:55:20,629 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. 2023-03-08 19:55:20,630 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 3/3 local region snapshots. 2023-03-08 19:55:20,630 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(314): Completed 3 local region snapshots. 2023-03-08 19:55:20,630 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(345): cancelling 0 tasks for snapshot jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:20,630 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'snaptb-1678305319718' locally completed 2023-03-08 19:55:20,630 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'snaptb-1678305319718' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:55:20,632 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'snaptb-1678305319718' has notified controller of completion 2023-03-08 19:55:20,632 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1678305319718/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:20,632 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:55:20,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/reached/snaptb-1678305319718/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:20,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:20,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-03-08 19:55:20,632 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'snaptb-1678305319718' completed. 2023-03-08 19:55:20,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:20,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:20,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678305319718 2023-03-08 19:55:20,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:20,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:20,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678305319718 2023-03-08 19:55:20,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:20,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'snaptb-1678305319718' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:55:20,634 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'snaptb-1678305319718', counting down latch. Waiting for 0 more 2023-03-08 19:55:20,634 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'snaptb-1678305319718' execution completed 2023-03-08 19:55:20,634 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:55:20,634 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:55:20,634 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:snaptb-1678305319718 2023-03-08 19:55:20,634 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure snaptb-1678305319718including nodes /hbase/online-snapshot/acquired /hbase/online-snapshot/reached /hbase/online-snapshot/abort 2023-03-08 19:55:20,636 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1678305319718 2023-03-08 19:55:20,636 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1678305319718 2023-03-08 19:55:20,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/abort/snaptb-1678305319718 2023-03-08 19:55:20,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/snaptb-1678305319718 2023-03-08 19:55:20,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1678305319718 2023-03-08 19:55:20,636 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-03-08 19:55:20,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:20,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-03-08 19:55:20,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-03-08 19:55:20,636 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/snaptb-1678305319718/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:20,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-03-08 19:55:20,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:20,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1678305319718 2023-03-08 19:55:20,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678305319718 2023-03-08 19:55:20,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:20,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678305319718 2023-03-08 19:55:20,637 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/snaptb-1678305319718/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:20,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:20,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:20,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678305319718 2023-03-08 19:55:20,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:20,640 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1678305319718/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:20,640 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-03-08 19:55:20,640 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:55:20,640 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(97): Done waiting - online snapshot for snaptb-1678305319718 2023-03-08 19:55:20,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-03-08 19:55:20,640 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(113): Taking snapshot for mob files in table testCompactionWithHFileLink 2023-03-08 19:55:20,640 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-03-08 19:55:20,641 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(191): Storing mob region '{ENCODED => 0e7b6fa9e81bd62588edd7d77d6165e4, NAME => 'testCompactionWithHFileLink,.mob,0.0e7b6fa9e81bd62588edd7d77d6165e4.', STARTKEY => '.mob', ENDKEY => ''}' region-info for snapshot=snaptb-1678305319718 2023-03-08 19:55:20,641 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/online-snapshot/abort/snaptb-1678305319718 because node does not exist (not an error) 2023-03-08 19:55:20,641 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(196): Creating references for mob files 2023-03-08 19:55:20,640 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1678305319718 2023-03-08 19:55:20,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-03-08 19:55:20,641 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1678305319718 2023-03-08 19:55:20,641 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1678305319718/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:20,641 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1678305319718 2023-03-08 19:55:20,641 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1678305319718 2023-03-08 19:55:20,641 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1678305319718 2023-03-08 19:55:20,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-03-08 19:55:20,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-03-08 19:55:20,642 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303083653e76c42154508bdba69de9d712f83, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303088ee4da827de14300899601e052c73946, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308d3c2ea31978640cf9022c0e5fb535d1c, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308e7a0a9d515b24f63a4013e970fb178a0, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030803a51c78c3a9414481e4489a8fc504ad, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303088ffe3f5d37194c0fb226cac523d21fef, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308c5433010db254bcfb902142789f105cc, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308dd1c4643f5654221b89fb3ab8d9a8eb4, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303088d68f70559bb4783aaa77c40b3a3e2c0, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303089f52416d158d4f62ac1989a0c5c86166, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308b87a29e85e7140f29524a1dd29216255, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308c0451f6624404cb9a261ce5d697fda74] mob files 2023-03-08 19:55:20,642 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303083653e76c42154508bdba69de9d712f83 2023-03-08 19:55:20,642 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303088ee4da827de14300899601e052c73946 2023-03-08 19:55:20,642 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308d3c2ea31978640cf9022c0e5fb535d1c 2023-03-08 19:55:20,642 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308e7a0a9d515b24f63a4013e970fb178a0 2023-03-08 19:55:20,642 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030803a51c78c3a9414481e4489a8fc504ad 2023-03-08 19:55:20,642 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303088ffe3f5d37194c0fb226cac523d21fef 2023-03-08 19:55:20,642 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308c5433010db254bcfb902142789f105cc 2023-03-08 19:55:20,642 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308dd1c4643f5654221b89fb3ab8d9a8eb4 2023-03-08 19:55:20,642 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303088d68f70559bb4783aaa77c40b3a3e2c0 2023-03-08 19:55:20,642 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303089f52416d158d4f62ac1989a0c5c86166 2023-03-08 19:55:20,642 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308b87a29e85e7140f29524a1dd29216255 2023-03-08 19:55:20,642 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308c0451f6624404cb9a261ce5d697fda74 2023-03-08 19:55:20,643 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308063437f66d8b4c478dc3e48cccd84c31, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023030876751d04d8044b56831dbe61c59ec030, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308b47202b6a5174e87a5ae0040bc3efca7, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308f529b81cd56b4bc38d7251edc066f179, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230308530634480de84dbdbb673bdb4d36dcfe, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303085a588abaab334b20b603cfd602cb2f9e, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303085c368878706a4756a425c85247df761f, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230308b5e613f2acc74fef836cd60172ea0d37, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308336c8ea8d5a648e5a58dd344250b0346, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303083e3cf9a6a99e4f1ca67c739bc1f51618, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023030843a51fd1ba354014a3638289006916d9, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308a8d8d630c1774bd8bbc29cb46a189584] mob files 2023-03-08 19:55:20,643 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308063437f66d8b4c478dc3e48cccd84c31 2023-03-08 19:55:20,643 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023030876751d04d8044b56831dbe61c59ec030 2023-03-08 19:55:20,643 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308b47202b6a5174e87a5ae0040bc3efca7 2023-03-08 19:55:20,643 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308f529b81cd56b4bc38d7251edc066f179 2023-03-08 19:55:20,643 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230308530634480de84dbdbb673bdb4d36dcfe 2023-03-08 19:55:20,643 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303085a588abaab334b20b603cfd602cb2f9e 2023-03-08 19:55:20,643 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303085c368878706a4756a425c85247df761f 2023-03-08 19:55:20,643 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230308b5e613f2acc74fef836cd60172ea0d37 2023-03-08 19:55:20,643 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308336c8ea8d5a648e5a58dd344250b0346 2023-03-08 19:55:20,644 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303083e3cf9a6a99e4f1ca67c739bc1f51618 2023-03-08 19:55:20,644 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023030843a51fd1ba354014a3638289006916d9 2023-03-08 19:55:20,644 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308a8d8d630c1774bd8bbc29cb46a189584 2023-03-08 19:55:20,871 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 19:55:20,872 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678305319718 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 19:55:20,872 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1678305319718 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-08 19:55:20,872 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-03-08 19:55:21,049 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for snaptb-1678305319718 2023-03-08 19:55:21,051 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.hbase-snapshot/.tmp/snaptb-1678305319718 2023-03-08 19:55:21,497 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.hbase-snapshot/.tmp/snaptb-1678305319718 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.hbase-snapshot/snaptb-1678305319718 2023-03-08 19:55:21,872 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 19:55:21,873 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678305319718 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 19:55:21,873 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1678305319718 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-08 19:55:21,873 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2428): (#6) Sleeping: 2000ms while waiting for snapshot completion. 2023-03-08 19:55:22,313 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot snaptb-1678305319718 of table testCompactionWithHFileLink completed 2023-03-08 19:55:22,313 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.hbase-snapshot/.tmp/snaptb-1678305319718 2023-03-08 19:55:22,314 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.hbase-snapshot/.tmp/snaptb-1678305319718 2023-03-08 19:55:22,314 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running FLUSH table snapshot snaptb-1678305319718 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1678305319756Completed referencing HFiles for the mob region of table: testCompactionWithHFileLink at 1678305321049 (+1293 ms)Consolidate snapshot: snaptb-1678305319718 at 1678305321049Loading Region manifests for snaptb-1678305319718 at 1678305321049Writing data manifest for snaptb-1678305319718 at 1678305321059 (+10 ms)Verifying snapshot: snaptb-1678305319718 at 1678305321467 (+408 ms)Snapshot snaptb-1678305319718 of table testCompactionWithHFileLink completed at 1678305322313 (+846 ms) 2023-03-08 19:55:22,315 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=87, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-03-08 19:55:22,316 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=87, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED in 2.5580 sec 2023-03-08 19:55:23,162 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 19:55:23,206 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5d9df324 to 127.0.0.1:56751 2023-03-08 19:55:23,206 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 19:55:23,206 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 19:55:23,206 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x12ee7b19 to 127.0.0.1:56751 2023-03-08 19:55:23,206 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 19:55:23,207 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 19:55:23,207 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x29a03c3d to 127.0.0.1:56751 2023-03-08 19:55:23,207 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 19:55:23,207 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 19:55:23,207 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x76bcf796 to 127.0.0.1:56751 2023-03-08 19:55:23,207 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 19:55:23,397 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(142): Chore: CompactedHFilesCleaner missed its start time 2023-03-08 19:55:23,873 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 19:55:23,874 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678305319718 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 19:55:23,875 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] snapshot.SnapshotManager(401): Snapshot '{ ss=snaptb-1678305319718 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-03-08 19:55:23,907 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:55:23,909 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-08 19:55:23,909 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-08 19:55:23,909 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-08 19:55:23,909 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:55:23,909 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-08 19:55:23,909 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-08 19:55:23,910 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:23,910 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 19:55:23,911 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:55:23,911 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:23,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:55:23,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:55:23,911 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:23,911 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:55:23,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 19:55:23,912 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:23,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:55:23,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 19:55:23,912 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-08 19:55:23,912 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-08 19:55:23,912 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-08 19:55:23,912 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:55:23,912 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-08 19:55:23,912 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. 2023-03-08 19:55:23,913 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. started... 2023-03-08 19:55:23,913 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. 2023-03-08 19:55:23,913 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. 2023-03-08 19:55:23,913 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. started... 2023-03-08 19:55:23,913 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 19:55:23,913 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing bf25ef5682cfde9aa90a5d6fe8229bdd 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-08 19:55:23,913 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:55:23,913 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9c99479945efcf910b16288e607c83eb 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-08 19:55:23,913 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. started... 2023-03-08 19:55:23,913 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing d392b5a609f9661af565e4c73dab47f8 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-08 19:55:24,330 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/.tmp/family1/7ec37243809e48828100491a8d2a759a 2023-03-08 19:55:24,333 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7ec37243809e48828100491a8d2a759a 2023-03-08 19:55:24,728 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/.tmp/family1/fe796592430e40e79cafe2164f9a1dd1 2023-03-08 19:55:24,730 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/.tmp/family1/fa9c43ba057c402ba4104df7d10a3351 2023-03-08 19:55:24,732 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fe796592430e40e79cafe2164f9a1dd1 2023-03-08 19:55:24,734 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fa9c43ba057c402ba4104df7d10a3351 2023-03-08 19:55:24,765 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/.tmp/family2/f4996a62000b4ddb8eb399ab1e6b0a1c 2023-03-08 19:55:24,769 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f4996a62000b4ddb8eb399ab1e6b0a1c 2023-03-08 19:55:24,770 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/.tmp/family1/7ec37243809e48828100491a8d2a759a as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/7ec37243809e48828100491a8d2a759a 2023-03-08 19:55:24,774 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7ec37243809e48828100491a8d2a759a 2023-03-08 19:55:24,774 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/7ec37243809e48828100491a8d2a759a, entries=3, sequenceid=24, filesize=5.0 K 2023-03-08 19:55:24,775 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/.tmp/family2/f4996a62000b4ddb8eb399ab1e6b0a1c as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/f4996a62000b4ddb8eb399ab1e6b0a1c 2023-03-08 19:55:24,779 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f4996a62000b4ddb8eb399ab1e6b0a1c 2023-03-08 19:55:24,779 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/f4996a62000b4ddb8eb399ab1e6b0a1c, entries=1, sequenceid=24, filesize=5.0 K 2023-03-08 19:55:24,780 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for bf25ef5682cfde9aa90a5d6fe8229bdd in 867ms, sequenceid=24, compaction requested=true 2023-03-08 19:55:24,780 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for bf25ef5682cfde9aa90a5d6fe8229bdd: 2023-03-08 19:55:24,780 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. 2023-03-08 19:55:24,780 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 19:55:25,156 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/.tmp/family2/9cc6c1df84bf4fe5931eaed6486cde40 2023-03-08 19:55:25,159 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9cc6c1df84bf4fe5931eaed6486cde40 2023-03-08 19:55:25,160 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/.tmp/family1/fa9c43ba057c402ba4104df7d10a3351 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/fa9c43ba057c402ba4104df7d10a3351 2023-03-08 19:55:25,163 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fa9c43ba057c402ba4104df7d10a3351 2023-03-08 19:55:25,163 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/fa9c43ba057c402ba4104df7d10a3351, entries=3, sequenceid=24, filesize=5.0 K 2023-03-08 19:55:25,164 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/.tmp/family2/9cc6c1df84bf4fe5931eaed6486cde40 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/9cc6c1df84bf4fe5931eaed6486cde40 2023-03-08 19:55:25,166 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/.tmp/family2/582c263da42b474ab618a306b7e37ff8 2023-03-08 19:55:25,168 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9cc6c1df84bf4fe5931eaed6486cde40 2023-03-08 19:55:25,168 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/9cc6c1df84bf4fe5931eaed6486cde40, entries=1, sequenceid=24, filesize=5.0 K 2023-03-08 19:55:25,169 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for d392b5a609f9661af565e4c73dab47f8 in 1256ms, sequenceid=24, compaction requested=true 2023-03-08 19:55:25,169 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for d392b5a609f9661af565e4c73dab47f8: 2023-03-08 19:55:25,169 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. 2023-03-08 19:55:25,169 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 19:55:25,169 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 582c263da42b474ab618a306b7e37ff8 2023-03-08 19:55:25,170 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/.tmp/family1/fe796592430e40e79cafe2164f9a1dd1 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/fe796592430e40e79cafe2164f9a1dd1 2023-03-08 19:55:25,173 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fe796592430e40e79cafe2164f9a1dd1 2023-03-08 19:55:25,173 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/fe796592430e40e79cafe2164f9a1dd1, entries=3, sequenceid=24, filesize=5.0 K 2023-03-08 19:55:25,174 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/.tmp/family2/582c263da42b474ab618a306b7e37ff8 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/582c263da42b474ab618a306b7e37ff8 2023-03-08 19:55:25,178 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 582c263da42b474ab618a306b7e37ff8 2023-03-08 19:55:25,178 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/582c263da42b474ab618a306b7e37ff8, entries=1, sequenceid=24, filesize=5.0 K 2023-03-08 19:55:25,179 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 9c99479945efcf910b16288e607c83eb in 1266ms, sequenceid=24, compaction requested=true 2023-03-08 19:55:25,179 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9c99479945efcf910b16288e607c83eb: 2023-03-08 19:55:25,179 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. 2023-03-08 19:55:25,179 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 19:55:25,179 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 19:55:25,179 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:25,179 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-08 19:55:25,179 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-08 19:55:25,181 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:25,181 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 19:55:25,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:25,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:25,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:25,181 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 19:55:25,181 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:55:25,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:25,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:25,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 19:55:25,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:25,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:25,182 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-08 19:55:25,182 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@587b4439[Count = 0] remaining members to acquire global barrier 2023-03-08 19:55:25,182 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-08 19:55:25,182 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 19:55:25,187 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 19:55:25,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 19:55:25,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 19:55:25,187 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-08 19:55:25,187 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-08 19:55:25,187 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:55:25,187 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:25,187 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:55:25,188 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:25,188 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-08 19:55:25,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:25,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:25,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:25,188 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:55:25,188 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-08 19:55:25,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:25,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:25,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 19:55:25,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:25,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:25,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 19:55:25,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:25,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:55:25,190 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-08 19:55:25,190 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-08 19:55:25,190 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:55:25,190 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:55:25,190 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-08 19:55:25,190 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:55:25,192 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:25,192 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:25,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:25,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:25,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:25,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:25,192 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:55:25,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:25,192 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:25,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:55:25,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:55:25,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:25,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:25,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 19:55:25,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:25,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 19:55:25,194 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:25,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:25,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:25,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-08 19:55:25,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:25,196 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:25,196 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:55:25,196 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 19:55:25,196 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:55:25,196 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-08 19:55:25,196 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:55:25,196 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:55:25,196 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-08 19:55:25,196 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:25,196 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 19:55:25,196 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-08 19:55:25,196 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-08 19:55:25,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:55:25,196 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-08 19:55:25,196 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-08 19:55:25,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:55:25,196 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:55:25,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:55:25,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:55:25,297 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:55:25,298 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:55:25,299 DEBUG [Listener at localhost/34049] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 19:55:25,299 DEBUG [Listener at localhost/34049] regionserver.HStore(1912): 9c99479945efcf910b16288e607c83eb/family1 is initiating major compaction (all files) 2023-03-08 19:55:25,299 INFO [Listener at localhost/34049] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 19:55:25,299 INFO [Listener at localhost/34049] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 19:55:25,300 INFO [Listener at localhost/34049] regionserver.HRegion(2259): Starting compaction of 9c99479945efcf910b16288e607c83eb/family1 in testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. 2023-03-08 19:55:25,300 INFO [Listener at localhost/34049] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/123a27e9c7024e02b0fbca054fd86280, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/a433c1e1bdb446f385d1156aa8209307, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/9a28a7cd1d794cde8d25ec94745b3bb9, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/f7d2cb4000d943219c0ae3c05e060127, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/fe796592430e40e79cafe2164f9a1dd1] into tmpdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/.tmp, totalSize=26.7 K 2023-03-08 19:55:25,301 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 123a27e9c7024e02b0fbca054fd86280, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678305313717 2023-03-08 19:55:25,301 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting a433c1e1bdb446f385d1156aa8209307, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678305315116 2023-03-08 19:55:25,302 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 9a28a7cd1d794cde8d25ec94745b3bb9, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678305316519 2023-03-08 19:55:25,302 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting f7d2cb4000d943219c0ae3c05e060127, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678305317923 2023-03-08 19:55:25,302 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting fe796592430e40e79cafe2164f9a1dd1, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-08 19:55:25,316 INFO [Listener at localhost/34049] throttle.PressureAwareThroughputController(145): 9c99479945efcf910b16288e607c83eb#family1#compaction#108 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 19:55:25,724 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023030866611143d4764f488460dc689a326efa_del 2023-03-08 19:55:25,724 INFO [Listener at localhost/34049] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023030866611143d4764f488460dc689a326efa_del to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023030866611143d4764f488460dc689a326efa_del 2023-03-08 19:55:26,145 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e7667af31c2b473a97ad03da49d7dc26 2023-03-08 19:55:26,146 DEBUG [Listener at localhost/34049] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/.tmp/family1/e7667af31c2b473a97ad03da49d7dc26 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/e7667af31c2b473a97ad03da49d7dc26 2023-03-08 19:55:26,149 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e7667af31c2b473a97ad03da49d7dc26 2023-03-08 19:55:26,150 INFO [Listener at localhost/34049] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 9c99479945efcf910b16288e607c83eb/family1 of 9c99479945efcf910b16288e607c83eb into e7667af31c2b473a97ad03da49d7dc26(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 19:55:26,150 DEBUG [Listener at localhost/34049] regionserver.HRegion(2289): Compaction status journal for 9c99479945efcf910b16288e607c83eb: 2023-03-08 19:55:26,150 DEBUG [Listener at localhost/34049] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 19:55:26,150 DEBUG [Listener at localhost/34049] regionserver.HStore(1912): 9c99479945efcf910b16288e607c83eb/family2 is initiating major compaction (all files) 2023-03-08 19:55:26,150 INFO [Listener at localhost/34049] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 19:55:26,150 INFO [Listener at localhost/34049] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 19:55:26,151 INFO [Listener at localhost/34049] regionserver.HRegion(2259): Starting compaction of 9c99479945efcf910b16288e607c83eb/family2 in testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. 2023-03-08 19:55:26,151 INFO [Listener at localhost/34049] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/cdaeaeb4819a41cc88f69b9970a33dcb, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/68353db2020f4adeba9aeb341538de89, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/7957d21df4a84e90b22385e13d00adde, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/3f39c3b5bff246788da3275e205043fb, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/582c263da42b474ab618a306b7e37ff8] into tmpdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/.tmp, totalSize=25.4 K 2023-03-08 19:55:26,152 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting cdaeaeb4819a41cc88f69b9970a33dcb, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678305313717 2023-03-08 19:55:26,152 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 68353db2020f4adeba9aeb341538de89, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678305315116 2023-03-08 19:55:26,152 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 7957d21df4a84e90b22385e13d00adde, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678305316519 2023-03-08 19:55:26,152 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 3f39c3b5bff246788da3275e205043fb, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678305317923 2023-03-08 19:55:26,153 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 582c263da42b474ab618a306b7e37ff8, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-08 19:55:26,164 INFO [Listener at localhost/34049] throttle.PressureAwareThroughputController(145): 9c99479945efcf910b16288e607c83eb#family2#compaction#109 average throughput is 0.35 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 19:55:26,198 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/123a27e9c7024e02b0fbca054fd86280, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/a433c1e1bdb446f385d1156aa8209307, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/9a28a7cd1d794cde8d25ec94745b3bb9, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/f7d2cb4000d943219c0ae3c05e060127, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/fe796592430e40e79cafe2164f9a1dd1] to archive 2023-03-08 19:55:26,199 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:55:26,200 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/123a27e9c7024e02b0fbca054fd86280 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/123a27e9c7024e02b0fbca054fd86280 2023-03-08 19:55:26,201 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/a433c1e1bdb446f385d1156aa8209307 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/a433c1e1bdb446f385d1156aa8209307 2023-03-08 19:55:26,202 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/9a28a7cd1d794cde8d25ec94745b3bb9 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/9a28a7cd1d794cde8d25ec94745b3bb9 2023-03-08 19:55:26,203 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/f7d2cb4000d943219c0ae3c05e060127 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/f7d2cb4000d943219c0ae3c05e060127 2023-03-08 19:55:26,204 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/fe796592430e40e79cafe2164f9a1dd1 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/fe796592430e40e79cafe2164f9a1dd1 2023-03-08 19:55:26,574 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202303089ba32d56247f467e90d3904a4b2aea47_del 2023-03-08 19:55:26,575 INFO [Listener at localhost/34049] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303089ba32d56247f467e90d3904a4b2aea47_del to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303089ba32d56247f467e90d3904a4b2aea47_del 2023-03-08 19:55:27,389 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3b972ece89c34d3691dcfb92a6dfa7d7 2023-03-08 19:55:27,390 DEBUG [Listener at localhost/34049] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/.tmp/family2/3b972ece89c34d3691dcfb92a6dfa7d7 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/3b972ece89c34d3691dcfb92a6dfa7d7 2023-03-08 19:55:27,394 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3b972ece89c34d3691dcfb92a6dfa7d7 2023-03-08 19:55:27,394 INFO [Listener at localhost/34049] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 9c99479945efcf910b16288e607c83eb/family2 of 9c99479945efcf910b16288e607c83eb into 3b972ece89c34d3691dcfb92a6dfa7d7(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 19:55:27,395 DEBUG [Listener at localhost/34049] regionserver.HRegion(2289): Compaction status journal for 9c99479945efcf910b16288e607c83eb: 2023-03-08 19:55:27,395 DEBUG [Listener at localhost/34049] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 19:55:27,395 DEBUG [Listener at localhost/34049] regionserver.HStore(1912): bf25ef5682cfde9aa90a5d6fe8229bdd/family1 is initiating major compaction (all files) 2023-03-08 19:55:27,395 INFO [Listener at localhost/34049] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 19:55:27,395 INFO [Listener at localhost/34049] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 19:55:27,396 INFO [Listener at localhost/34049] regionserver.HRegion(2259): Starting compaction of bf25ef5682cfde9aa90a5d6fe8229bdd/family1 in testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. 2023-03-08 19:55:27,396 INFO [Listener at localhost/34049] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/8cb2d3dd62cc42009c2420fabc400fb6, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/af1975c5b195440797e41fe2eba32cc4, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/046ec2868b184c3e84b6105bf189d3bd, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/c9ebe20caf5d4c2cb57adb8d0820d554, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/7ec37243809e48828100491a8d2a759a] into tmpdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/.tmp, totalSize=26.7 K 2023-03-08 19:55:27,396 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 8cb2d3dd62cc42009c2420fabc400fb6, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678305313716 2023-03-08 19:55:27,396 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting af1975c5b195440797e41fe2eba32cc4, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678305315116 2023-03-08 19:55:27,397 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 046ec2868b184c3e84b6105bf189d3bd, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678305316519 2023-03-08 19:55:27,397 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting c9ebe20caf5d4c2cb57adb8d0820d554, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678305317923 2023-03-08 19:55:27,397 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 7ec37243809e48828100491a8d2a759a, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-08 19:55:27,398 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/cdaeaeb4819a41cc88f69b9970a33dcb, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/68353db2020f4adeba9aeb341538de89, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/7957d21df4a84e90b22385e13d00adde, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/3f39c3b5bff246788da3275e205043fb, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/582c263da42b474ab618a306b7e37ff8] to archive 2023-03-08 19:55:27,398 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:55:27,400 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/cdaeaeb4819a41cc88f69b9970a33dcb to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/cdaeaeb4819a41cc88f69b9970a33dcb 2023-03-08 19:55:27,401 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/68353db2020f4adeba9aeb341538de89 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/68353db2020f4adeba9aeb341538de89 2023-03-08 19:55:27,402 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/7957d21df4a84e90b22385e13d00adde to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/7957d21df4a84e90b22385e13d00adde 2023-03-08 19:55:27,404 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/3f39c3b5bff246788da3275e205043fb to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/3f39c3b5bff246788da3275e205043fb 2023-03-08 19:55:27,405 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/582c263da42b474ab618a306b7e37ff8 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/582c263da42b474ab618a306b7e37ff8 2023-03-08 19:55:27,409 INFO [Listener at localhost/34049] throttle.PressureAwareThroughputController(145): bf25ef5682cfde9aa90a5d6fe8229bdd#family1#compaction#110 average throughput is 0.85 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 19:55:27,822 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230308e36338f004c048a7af634c9f1c63d2e3_del 2023-03-08 19:55:27,823 INFO [Listener at localhost/34049] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308e36338f004c048a7af634c9f1c63d2e3_del to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308e36338f004c048a7af634c9f1c63d2e3_del 2023-03-08 19:55:28,637 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e7702e28b4254a2ab4f1ba997055c5a7 2023-03-08 19:55:28,638 DEBUG [Listener at localhost/34049] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/.tmp/family1/e7702e28b4254a2ab4f1ba997055c5a7 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/e7702e28b4254a2ab4f1ba997055c5a7 2023-03-08 19:55:28,642 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e7702e28b4254a2ab4f1ba997055c5a7 2023-03-08 19:55:28,643 INFO [Listener at localhost/34049] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in bf25ef5682cfde9aa90a5d6fe8229bdd/family1 of bf25ef5682cfde9aa90a5d6fe8229bdd into e7702e28b4254a2ab4f1ba997055c5a7(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 19:55:28,644 DEBUG [Listener at localhost/34049] regionserver.HRegion(2289): Compaction status journal for bf25ef5682cfde9aa90a5d6fe8229bdd: 2023-03-08 19:55:28,644 DEBUG [Listener at localhost/34049] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 19:55:28,644 DEBUG [Listener at localhost/34049] regionserver.HStore(1912): bf25ef5682cfde9aa90a5d6fe8229bdd/family2 is initiating major compaction (all files) 2023-03-08 19:55:28,644 INFO [Listener at localhost/34049] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 19:55:28,644 INFO [Listener at localhost/34049] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 19:55:28,644 INFO [Listener at localhost/34049] regionserver.HRegion(2259): Starting compaction of bf25ef5682cfde9aa90a5d6fe8229bdd/family2 in testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. 2023-03-08 19:55:28,644 INFO [Listener at localhost/34049] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/044f1e6d183045beae5b75645bb97f60, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/d9b3dfdcafcf4aa3b87dfdc6c038bf9e, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/d97fbd02a2e04d4cb97a3bfe7772fb6e, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/e323979e94844e49a28882288ab3a208, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/f4996a62000b4ddb8eb399ab1e6b0a1c] into tmpdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/.tmp, totalSize=25.4 K 2023-03-08 19:55:28,645 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 044f1e6d183045beae5b75645bb97f60, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678305313716 2023-03-08 19:55:28,645 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting d9b3dfdcafcf4aa3b87dfdc6c038bf9e, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678305315116 2023-03-08 19:55:28,646 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting d97fbd02a2e04d4cb97a3bfe7772fb6e, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678305316519 2023-03-08 19:55:28,646 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting e323979e94844e49a28882288ab3a208, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678305317923 2023-03-08 19:55:28,647 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting f4996a62000b4ddb8eb399ab1e6b0a1c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-08 19:55:28,660 INFO [Listener at localhost/34049] throttle.PressureAwareThroughputController(145): bf25ef5682cfde9aa90a5d6fe8229bdd#family2#compaction#111 average throughput is 0.26 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 19:55:28,702 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/8cb2d3dd62cc42009c2420fabc400fb6, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/af1975c5b195440797e41fe2eba32cc4, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/046ec2868b184c3e84b6105bf189d3bd, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/c9ebe20caf5d4c2cb57adb8d0820d554, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/7ec37243809e48828100491a8d2a759a] to archive 2023-03-08 19:55:28,703 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:55:28,704 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/8cb2d3dd62cc42009c2420fabc400fb6 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/8cb2d3dd62cc42009c2420fabc400fb6 2023-03-08 19:55:28,706 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/af1975c5b195440797e41fe2eba32cc4 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/af1975c5b195440797e41fe2eba32cc4 2023-03-08 19:55:28,706 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/046ec2868b184c3e84b6105bf189d3bd to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/046ec2868b184c3e84b6105bf189d3bd 2023-03-08 19:55:28,707 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/c9ebe20caf5d4c2cb57adb8d0820d554 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/c9ebe20caf5d4c2cb57adb8d0820d554 2023-03-08 19:55:28,708 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/7ec37243809e48828100491a8d2a759a to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/7ec37243809e48828100491a8d2a759a 2023-03-08 19:55:29,069 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202303083c55225899294a1a951e8ea7bdb2092b_del 2023-03-08 19:55:29,069 INFO [Listener at localhost/34049] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303083c55225899294a1a951e8ea7bdb2092b_del to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303083c55225899294a1a951e8ea7bdb2092b_del 2023-03-08 19:55:29,883 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for edf3f0111126479b983a6c0fdea5b711 2023-03-08 19:55:29,883 DEBUG [Listener at localhost/34049] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/.tmp/family2/edf3f0111126479b983a6c0fdea5b711 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/edf3f0111126479b983a6c0fdea5b711 2023-03-08 19:55:29,887 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for edf3f0111126479b983a6c0fdea5b711 2023-03-08 19:55:29,888 INFO [Listener at localhost/34049] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in bf25ef5682cfde9aa90a5d6fe8229bdd/family2 of bf25ef5682cfde9aa90a5d6fe8229bdd into edf3f0111126479b983a6c0fdea5b711(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 19:55:29,888 DEBUG [Listener at localhost/34049] regionserver.HRegion(2289): Compaction status journal for bf25ef5682cfde9aa90a5d6fe8229bdd: 2023-03-08 19:55:29,888 DEBUG [Listener at localhost/34049] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 19:55:29,889 DEBUG [Listener at localhost/34049] regionserver.HStore(1912): d392b5a609f9661af565e4c73dab47f8/family1 is initiating major compaction (all files) 2023-03-08 19:55:29,889 INFO [Listener at localhost/34049] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 19:55:29,889 INFO [Listener at localhost/34049] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 19:55:29,889 INFO [Listener at localhost/34049] regionserver.HRegion(2259): Starting compaction of d392b5a609f9661af565e4c73dab47f8/family1 in testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. 2023-03-08 19:55:29,889 INFO [Listener at localhost/34049] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/7372b15acde5432d9949bd6d62365bd8, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/e39643d9b2424962b7f3883e1e919f9a, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/c285d649295f485d986c468697edc0ce, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/67ebbe2e123e43d18dd53054062690ec, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/fa9c43ba057c402ba4104df7d10a3351] into tmpdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/.tmp, totalSize=26.7 K 2023-03-08 19:55:29,889 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 7372b15acde5432d9949bd6d62365bd8, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678305313718 2023-03-08 19:55:29,890 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting e39643d9b2424962b7f3883e1e919f9a, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678305315116 2023-03-08 19:55:29,890 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting c285d649295f485d986c468697edc0ce, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678305316519 2023-03-08 19:55:29,890 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 67ebbe2e123e43d18dd53054062690ec, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678305317923 2023-03-08 19:55:29,891 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting fa9c43ba057c402ba4104df7d10a3351, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-08 19:55:29,899 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/044f1e6d183045beae5b75645bb97f60, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/d9b3dfdcafcf4aa3b87dfdc6c038bf9e, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/d97fbd02a2e04d4cb97a3bfe7772fb6e, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/e323979e94844e49a28882288ab3a208, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/f4996a62000b4ddb8eb399ab1e6b0a1c] to archive 2023-03-08 19:55:29,900 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:55:29,902 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/044f1e6d183045beae5b75645bb97f60 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/044f1e6d183045beae5b75645bb97f60 2023-03-08 19:55:29,904 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/d9b3dfdcafcf4aa3b87dfdc6c038bf9e to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/d9b3dfdcafcf4aa3b87dfdc6c038bf9e 2023-03-08 19:55:29,905 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/d97fbd02a2e04d4cb97a3bfe7772fb6e to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/d97fbd02a2e04d4cb97a3bfe7772fb6e 2023-03-08 19:55:29,906 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/e323979e94844e49a28882288ab3a208 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/e323979e94844e49a28882288ab3a208 2023-03-08 19:55:29,908 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/f4996a62000b4ddb8eb399ab1e6b0a1c to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/f4996a62000b4ddb8eb399ab1e6b0a1c 2023-03-08 19:55:29,912 INFO [Listener at localhost/34049] throttle.PressureAwareThroughputController(145): d392b5a609f9661af565e4c73dab47f8#family1#compaction#112 average throughput is 0.16 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 19:55:30,321 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202303085ef3ea96b9cd453a81015808954a7d41_del 2023-03-08 19:55:30,321 INFO [Listener at localhost/34049] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303085ef3ea96b9cd453a81015808954a7d41_del to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303085ef3ea96b9cd453a81015808954a7d41_del 2023-03-08 19:55:31,135 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0eb386651f534beda10396c53bbcc27e 2023-03-08 19:55:31,136 DEBUG [Listener at localhost/34049] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/.tmp/family1/0eb386651f534beda10396c53bbcc27e as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/0eb386651f534beda10396c53bbcc27e 2023-03-08 19:55:31,139 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0eb386651f534beda10396c53bbcc27e 2023-03-08 19:55:31,140 INFO [Listener at localhost/34049] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in d392b5a609f9661af565e4c73dab47f8/family1 of d392b5a609f9661af565e4c73dab47f8 into 0eb386651f534beda10396c53bbcc27e(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 19:55:31,140 DEBUG [Listener at localhost/34049] regionserver.HRegion(2289): Compaction status journal for d392b5a609f9661af565e4c73dab47f8: 2023-03-08 19:55:31,140 DEBUG [Listener at localhost/34049] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 19:55:31,141 DEBUG [Listener at localhost/34049] regionserver.HStore(1912): d392b5a609f9661af565e4c73dab47f8/family2 is initiating major compaction (all files) 2023-03-08 19:55:31,141 INFO [Listener at localhost/34049] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 19:55:31,141 INFO [Listener at localhost/34049] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 19:55:31,141 INFO [Listener at localhost/34049] regionserver.HRegion(2259): Starting compaction of d392b5a609f9661af565e4c73dab47f8/family2 in testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. 2023-03-08 19:55:31,141 INFO [Listener at localhost/34049] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/07773024718d41078416d0bd2e154765, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/3e46bffcb555430ca182ded732797e54, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/6eeb48f439a34bc9ae7493396a8ad303, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/a84f2cc20b89462cab9556cca240ba20, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/9cc6c1df84bf4fe5931eaed6486cde40] into tmpdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/.tmp, totalSize=25.4 K 2023-03-08 19:55:31,141 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 07773024718d41078416d0bd2e154765, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678305313718 2023-03-08 19:55:31,142 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 3e46bffcb555430ca182ded732797e54, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678305315116 2023-03-08 19:55:31,142 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 6eeb48f439a34bc9ae7493396a8ad303, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678305316519 2023-03-08 19:55:31,142 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting a84f2cc20b89462cab9556cca240ba20, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678305317923 2023-03-08 19:55:31,143 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 9cc6c1df84bf4fe5931eaed6486cde40, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-08 19:55:31,152 INFO [Listener at localhost/34049] throttle.PressureAwareThroughputController(145): d392b5a609f9661af565e4c73dab47f8#family2#compaction#113 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 19:55:31,199 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/7372b15acde5432d9949bd6d62365bd8, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/e39643d9b2424962b7f3883e1e919f9a, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/c285d649295f485d986c468697edc0ce, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/67ebbe2e123e43d18dd53054062690ec, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/fa9c43ba057c402ba4104df7d10a3351] to archive 2023-03-08 19:55:31,200 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:55:31,202 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/7372b15acde5432d9949bd6d62365bd8 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/7372b15acde5432d9949bd6d62365bd8 2023-03-08 19:55:31,203 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/e39643d9b2424962b7f3883e1e919f9a to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/e39643d9b2424962b7f3883e1e919f9a 2023-03-08 19:55:31,204 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/c285d649295f485d986c468697edc0ce to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/c285d649295f485d986c468697edc0ce 2023-03-08 19:55:31,205 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/67ebbe2e123e43d18dd53054062690ec to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/67ebbe2e123e43d18dd53054062690ec 2023-03-08 19:55:31,206 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/fa9c43ba057c402ba4104df7d10a3351 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/fa9c43ba057c402ba4104df7d10a3351 2023-03-08 19:55:31,560 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230308f902a5c46446458782d4c156522990a4_del 2023-03-08 19:55:31,560 INFO [Listener at localhost/34049] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308f902a5c46446458782d4c156522990a4_del to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230308f902a5c46446458782d4c156522990a4_del 2023-03-08 19:55:32,374 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7249207a38cc4f1bb9427f2354088baa 2023-03-08 19:55:32,375 DEBUG [Listener at localhost/34049] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/.tmp/family2/7249207a38cc4f1bb9427f2354088baa as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/7249207a38cc4f1bb9427f2354088baa 2023-03-08 19:55:32,378 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7249207a38cc4f1bb9427f2354088baa 2023-03-08 19:55:32,379 INFO [Listener at localhost/34049] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in d392b5a609f9661af565e4c73dab47f8/family2 of d392b5a609f9661af565e4c73dab47f8 into 7249207a38cc4f1bb9427f2354088baa(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 19:55:32,379 DEBUG [Listener at localhost/34049] regionserver.HRegion(2289): Compaction status journal for d392b5a609f9661af565e4c73dab47f8: 2023-03-08 19:55:32,384 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 19:55:32,388 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 19:55:32,392 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 19:55:32,393 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 19:55:32,402 INFO [Listener at localhost/34049] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-08 19:55:32,407 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/07773024718d41078416d0bd2e154765, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/3e46bffcb555430ca182ded732797e54, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/6eeb48f439a34bc9ae7493396a8ad303, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/a84f2cc20b89462cab9556cca240ba20, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/9cc6c1df84bf4fe5931eaed6486cde40] to archive 2023-03-08 19:55:32,408 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:55:32,410 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/07773024718d41078416d0bd2e154765 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/07773024718d41078416d0bd2e154765 2023-03-08 19:55:32,411 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/3e46bffcb555430ca182ded732797e54 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/3e46bffcb555430ca182ded732797e54 2023-03-08 19:55:32,412 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/6eeb48f439a34bc9ae7493396a8ad303 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/6eeb48f439a34bc9ae7493396a8ad303 2023-03-08 19:55:32,413 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/a84f2cc20b89462cab9556cca240ba20 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/a84f2cc20b89462cab9556cca240ba20 2023-03-08 19:55:32,415 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/9cc6c1df84bf4fe5931eaed6486cde40 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/9cc6c1df84bf4fe5931eaed6486cde40 2023-03-08 19:55:32,435 INFO [Listener at localhost/34049] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-03-08 19:55:32,437 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230308e36338f004c048a7af634c9f1c63d2e3_del 2023-03-08 19:55:32,440 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023030866611143d4764f488460dc689a326efa_del 2023-03-08 19:55:32,442 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202303085ef3ea96b9cd453a81015808954a7d41_del 2023-03-08 19:55:32,442 INFO [Listener at localhost/34049] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testCompactionWithHFileLink' column='family1' 2023-03-08 19:55:32,443 DEBUG [Listener at localhost/34049] zookeeper.ReadOnlyZKClient(139): Connect 0x00d77f15 to 127.0.0.1:56751 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 19:55:32,449 DEBUG [Listener at localhost/34049] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1716141, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 19:55:32,450 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 19:55:32,450 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 19:55:32,450 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 19:55:32,544 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308b6b34034f725406e8ec26fd16680d6bd to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308b6b34034f725406e8ec26fd16680d6bd 2023-03-08 19:55:32,544 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303080a5dec39dc3e4ef89f64b30dc2fed576 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303080a5dec39dc3e4ef89f64b30dc2fed576 2023-03-08 19:55:32,545 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 19:55:32,545 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 19:55:32,558 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 19:55:32,562 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33350, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 19:55:32,570 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 19:55:32,572 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37068, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 19:55:32,590 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230308/family1/6e9e845a53d04c30887a0977236090c3 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-03-08 19:55:32,591 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd., hostname=jenkins-hbase4.apache.org,33367,1678305218680, seqNum=2 for row with hfile group [{family1,hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230308/family1/6e9e845a53d04c30887a0977236090c3}] 2023-03-08 19:55:32,592 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230308/family1/d92eff374ed44abcbcd693bf96fac2d2 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-03-08 19:55:32,595 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb., hostname=jenkins-hbase4.apache.org,33367,1678305218680, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230308/family1/d92eff374ed44abcbcd693bf96fac2d2}] 2023-03-08 19:55:32,603 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(763): Validating hfile at hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230308/family1/d92eff374ed44abcbcd693bf96fac2d2 for inclusion in 9c99479945efcf910b16288e607c83eb/family1 2023-03-08 19:55:32,603 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HStore(763): Validating hfile at hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230308/family1/6e9e845a53d04c30887a0977236090c3 for inclusion in bf25ef5682cfde9aa90a5d6fe8229bdd/family1 2023-03-08 19:55:32,606 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-03-08 19:55:32,606 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-08 19:55:32,606 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(2446): Flush status journal for 9c99479945efcf910b16288e607c83eb: 2023-03-08 19:55:32,606 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-03-08 19:55:32,606 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HStore(777): Region bounds: first= last=1 2023-03-08 19:55:32,606 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HRegion(2446): Flush status journal for bf25ef5682cfde9aa90a5d6fe8229bdd: 2023-03-08 19:55:32,607 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230308/family1/d92eff374ed44abcbcd693bf96fac2d2 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testCompactionWithHFileLink__g7bfe87fqs0ac9d7vvl6rbbhsn0m2s87kilemrdd0s1459sg42ndoeolgrd4iu0l/family1/d92eff374ed44abcbcd693bf96fac2d2 2023-03-08 19:55:32,607 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230308/family1/6e9e845a53d04c30887a0977236090c3 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testCompactionWithHFileLink__51lgj1gnc7fjsttkdmei884mk5k76k44730vko9bgda1b75s5jt819fu1lkkf2f2/family1/6e9e845a53d04c30887a0977236090c3 2023-03-08 19:55:32,610 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testCompactionWithHFileLink__g7bfe87fqs0ac9d7vvl6rbbhsn0m2s87kilemrdd0s1459sg42ndoeolgrd4iu0l/family1/d92eff374ed44abcbcd693bf96fac2d2 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/3fca5386e0754389ae4e095454100867_SeqId_29_ 2023-03-08 19:55:32,610 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testCompactionWithHFileLink__51lgj1gnc7fjsttkdmei884mk5k76k44730vko9bgda1b75s5jt819fu1lkkf2f2/family1/6e9e845a53d04c30887a0977236090c3 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/89529938c0524656a66d7777082452c3_SeqId_29_ 2023-03-08 19:55:32,610 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(849): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testCompactionWithHFileLink__g7bfe87fqs0ac9d7vvl6rbbhsn0m2s87kilemrdd0s1459sg42ndoeolgrd4iu0l/family1/d92eff374ed44abcbcd693bf96fac2d2 into 9c99479945efcf910b16288e607c83eb/family1 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/3fca5386e0754389ae4e095454100867_SeqId_29_ - updating store file list. 2023-03-08 19:55:32,611 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HStore(849): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testCompactionWithHFileLink__51lgj1gnc7fjsttkdmei884mk5k76k44730vko9bgda1b75s5jt819fu1lkkf2f2/family1/6e9e845a53d04c30887a0977236090c3 into bf25ef5682cfde9aa90a5d6fe8229bdd/family1 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/89529938c0524656a66d7777082452c3_SeqId_29_ - updating store file list. 2023-03-08 19:55:32,615 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(882): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/3fca5386e0754389ae4e095454100867_SeqId_29_ into 9c99479945efcf910b16288e607c83eb/family1 2023-03-08 19:55:32,615 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(855): Successfully loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testCompactionWithHFileLink__g7bfe87fqs0ac9d7vvl6rbbhsn0m2s87kilemrdd0s1459sg42ndoeolgrd4iu0l/family1/d92eff374ed44abcbcd693bf96fac2d2 into 9c99479945efcf910b16288e607c83eb/family1 (new location: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/3fca5386e0754389ae4e095454100867_SeqId_29_) 2023-03-08 19:55:32,615 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HStore(882): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/89529938c0524656a66d7777082452c3_SeqId_29_ into bf25ef5682cfde9aa90a5d6fe8229bdd/family1 2023-03-08 19:55:32,616 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testCompactionWithHFileLink__g7bfe87fqs0ac9d7vvl6rbbhsn0m2s87kilemrdd0s1459sg42ndoeolgrd4iu0l/family1/d92eff374ed44abcbcd693bf96fac2d2 2023-03-08 19:55:32,616 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HStore(855): Successfully loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testCompactionWithHFileLink__51lgj1gnc7fjsttkdmei884mk5k76k44730vko9bgda1b75s5jt819fu1lkkf2f2/family1/6e9e845a53d04c30887a0977236090c3 into bf25ef5682cfde9aa90a5d6fe8229bdd/family1 (new location: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/89529938c0524656a66d7777082452c3_SeqId_29_) 2023-03-08 19:55:32,616 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testCompactionWithHFileLink__51lgj1gnc7fjsttkdmei884mk5k76k44730vko9bgda1b75s5jt819fu1lkkf2f2/family1/6e9e845a53d04c30887a0977236090c3 2023-03-08 19:55:32,618 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 19:55:32,618 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 19:55:32,618 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:55:32,619 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:55:32,620 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303083653e76c42154508bdba69de9d712f83 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303083653e76c42154508bdba69de9d712f83 2023-03-08 19:55:32,620 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303088d68f70559bb4783aaa77c40b3a3e2c0 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303088d68f70559bb4783aaa77c40b3a3e2c0 2023-03-08 19:55:32,621 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303088ee4da827de14300899601e052c73946 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303088ee4da827de14300899601e052c73946 2023-03-08 19:55:32,621 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303089f52416d158d4f62ac1989a0c5c86166 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303089f52416d158d4f62ac1989a0c5c86166 2023-03-08 19:55:32,623 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308d3c2ea31978640cf9022c0e5fb535d1c to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308d3c2ea31978640cf9022c0e5fb535d1c 2023-03-08 19:55:32,623 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308b87a29e85e7140f29524a1dd29216255 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308b87a29e85e7140f29524a1dd29216255 2023-03-08 19:55:32,624 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308c0451f6624404cb9a261ce5d697fda74 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308c0451f6624404cb9a261ce5d697fda74 2023-03-08 19:55:32,624 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-08 19:55:32,625 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230308' 2023-03-08 19:55:32,628 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308e7a0a9d515b24f63a4013e970fb178a0 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230308e7a0a9d515b24f63a4013e970fb178a0 2023-03-08 19:55:32,628 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-08 19:55:32,628 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230308' 2023-03-08 19:55:32,628 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 19:55:32,628 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 19:55:33,334 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023030811e4bcff5d214acaabd0b534ff5d326f to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030811e4bcff5d214acaabd0b534ff5d326f 2023-03-08 19:55:33,334 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 19:55:33,358 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230308/family1/82548d5c4d8b4bdeae770f2e74696a32 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-03-08 19:55:33,358 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8., hostname=jenkins-hbase4.apache.org,33367,1678305218680, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230308/family1/82548d5c4d8b4bdeae770f2e74696a32}] 2023-03-08 19:55:33,360 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(763): Validating hfile at hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230308/family1/82548d5c4d8b4bdeae770f2e74696a32 for inclusion in d392b5a609f9661af565e4c73dab47f8/family1 2023-03-08 19:55:33,362 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-03-08 19:55:33,362 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-08 19:55:33,362 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(2446): Flush status journal for d392b5a609f9661af565e4c73dab47f8: 2023-03-08 19:55:33,364 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230308/family1/82548d5c4d8b4bdeae770f2e74696a32 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testCompactionWithHFileLink__joh0ci9cov443s7449tl2urbcla0an40e9mehf8kr5l23g2g8qptm2vtjc5f77k5/family1/82548d5c4d8b4bdeae770f2e74696a32 2023-03-08 19:55:33,365 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testCompactionWithHFileLink__joh0ci9cov443s7449tl2urbcla0an40e9mehf8kr5l23g2g8qptm2vtjc5f77k5/family1/82548d5c4d8b4bdeae770f2e74696a32 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/85f0986f81a84e1888f18162717e0b79_SeqId_29_ 2023-03-08 19:55:33,366 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(849): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testCompactionWithHFileLink__joh0ci9cov443s7449tl2urbcla0an40e9mehf8kr5l23g2g8qptm2vtjc5f77k5/family1/82548d5c4d8b4bdeae770f2e74696a32 into d392b5a609f9661af565e4c73dab47f8/family1 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/85f0986f81a84e1888f18162717e0b79_SeqId_29_ - updating store file list. 2023-03-08 19:55:33,369 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(882): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/85f0986f81a84e1888f18162717e0b79_SeqId_29_ into d392b5a609f9661af565e4c73dab47f8/family1 2023-03-08 19:55:33,369 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(855): Successfully loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testCompactionWithHFileLink__joh0ci9cov443s7449tl2urbcla0an40e9mehf8kr5l23g2g8qptm2vtjc5f77k5/family1/82548d5c4d8b4bdeae770f2e74696a32 into d392b5a609f9661af565e4c73dab47f8/family1 (new location: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/85f0986f81a84e1888f18162717e0b79_SeqId_29_) 2023-03-08 19:55:33,369 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testCompactionWithHFileLink__joh0ci9cov443s7449tl2urbcla0an40e9mehf8kr5l23g2g8qptm2vtjc5f77k5/family1/82548d5c4d8b4bdeae770f2e74696a32 2023-03-08 19:55:33,371 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 19:55:33,371 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:55:33,372 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030803a51c78c3a9414481e4489a8fc504ad to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030803a51c78c3a9414481e4489a8fc504ad 2023-03-08 19:55:33,373 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303088ffe3f5d37194c0fb226cac523d21fef to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303088ffe3f5d37194c0fb226cac523d21fef 2023-03-08 19:55:33,374 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308c5433010db254bcfb902142789f105cc to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308c5433010db254bcfb902142789f105cc 2023-03-08 19:55:33,375 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308dd1c4643f5654221b89fb3ab8d9a8eb4 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308dd1c4643f5654221b89fb3ab8d9a8eb4 2023-03-08 19:55:33,375 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-08 19:55:33,375 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230308' 2023-03-08 19:55:33,375 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 19:55:33,375 INFO [Listener at localhost/34049] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-03-08 19:55:33,375 INFO [Listener at localhost/34049] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 19:55:33,375 INFO [Listener at localhost/34049] compactions.PartitionedMobCompactor(367): [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308e36338f004c048a7af634c9f1c63d2e3_del] 2023-03-08 19:55:33,376 DEBUG [Listener at localhost/34049] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:55:33,377 DEBUG [Listener at localhost/34049] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308e36338f004c048a7af634c9f1c63d2e3_del to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308e36338f004c048a7af634c9f1c63d2e3_del 2023-03-08 19:55:33,377 INFO [Listener at localhost/34049] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-03-08 19:55:33,377 INFO [Listener at localhost/34049] compactions.PartitionedMobCompactor(367): [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023030866611143d4764f488460dc689a326efa_del] 2023-03-08 19:55:33,377 DEBUG [Listener at localhost/34049] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:55:33,378 DEBUG [Listener at localhost/34049] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023030866611143d4764f488460dc689a326efa_del to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023030866611143d4764f488460dc689a326efa_del 2023-03-08 19:55:33,379 INFO [Listener at localhost/34049] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-03-08 19:55:33,379 INFO [Listener at localhost/34049] compactions.PartitionedMobCompactor(367): [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303085ef3ea96b9cd453a81015808954a7d41_del] 2023-03-08 19:55:33,379 DEBUG [Listener at localhost/34049] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:55:33,380 DEBUG [Listener at localhost/34049] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303085ef3ea96b9cd453a81015808954a7d41_del to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303085ef3ea96b9cd453a81015808954a7d41_del 2023-03-08 19:55:33,380 INFO [Listener at localhost/34049] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-03-08 19:55:33,383 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 19:55:33,386 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 19:55:33,389 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 19:55:33,389 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 19:55:33,395 INFO [Listener at localhost/34049] client.HBaseAdmin$15(890): Started disable of testCompactionWithHFileLink 2023-03-08 19:55:33,396 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testCompactionWithHFileLink 2023-03-08 19:55:33,396 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testCompactionWithHFileLink 2023-03-08 19:55:33,398 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-03-08 19:55:33,399 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678305333399"}]},"ts":"1678305333399"} 2023-03-08 19:55:33,400 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLING in hbase:meta 2023-03-08 19:55:33,401 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testCompactionWithHFileLink to state=DISABLING 2023-03-08 19:55:33,402 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bf25ef5682cfde9aa90a5d6fe8229bdd, UNASSIGN}, {pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=9c99479945efcf910b16288e607c83eb, UNASSIGN}, {pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d392b5a609f9661af565e4c73dab47f8, UNASSIGN}] 2023-03-08 19:55:33,403 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=9c99479945efcf910b16288e607c83eb, UNASSIGN 2023-03-08 19:55:33,403 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bf25ef5682cfde9aa90a5d6fe8229bdd, UNASSIGN 2023-03-08 19:55:33,403 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d392b5a609f9661af565e4c73dab47f8, UNASSIGN 2023-03-08 19:55:33,404 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=9c99479945efcf910b16288e607c83eb, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:33,404 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=bf25ef5682cfde9aa90a5d6fe8229bdd, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:33,404 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=d392b5a609f9661af565e4c73dab47f8, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:33,404 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678305333404"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678305333404"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678305333404"}]},"ts":"1678305333404"} 2023-03-08 19:55:33,404 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678305333404"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678305333404"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678305333404"}]},"ts":"1678305333404"} 2023-03-08 19:55:33,404 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678305333404"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678305333404"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678305333404"}]},"ts":"1678305333404"} 2023-03-08 19:55:33,405 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=91, state=RUNNABLE; CloseRegionProcedure d392b5a609f9661af565e4c73dab47f8, server=jenkins-hbase4.apache.org,33367,1678305218680}] 2023-03-08 19:55:33,406 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=90, state=RUNNABLE; CloseRegionProcedure 9c99479945efcf910b16288e607c83eb, server=jenkins-hbase4.apache.org,33367,1678305218680}] 2023-03-08 19:55:33,406 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=89, state=RUNNABLE; CloseRegionProcedure bf25ef5682cfde9aa90a5d6fe8229bdd, server=jenkins-hbase4.apache.org,33367,1678305218680}] 2023-03-08 19:55:33,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-03-08 19:55:33,558 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close d392b5a609f9661af565e4c73dab47f8 2023-03-08 19:55:33,558 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d392b5a609f9661af565e4c73dab47f8, disabling compactions & flushes 2023-03-08 19:55:33,558 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. 2023-03-08 19:55:33,558 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. 2023-03-08 19:55:33,558 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. after waiting 0 ms 2023-03-08 19:55:33,558 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. 2023-03-08 19:55:33,562 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-03-08 19:55:33,563 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. 2023-03-08 19:55:33,563 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d392b5a609f9661af565e4c73dab47f8: 2023-03-08 19:55:33,565 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed d392b5a609f9661af565e4c73dab47f8 2023-03-08 19:55:33,565 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close bf25ef5682cfde9aa90a5d6fe8229bdd 2023-03-08 19:55:33,565 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing bf25ef5682cfde9aa90a5d6fe8229bdd, disabling compactions & flushes 2023-03-08 19:55:33,565 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. 2023-03-08 19:55:33,565 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. 2023-03-08 19:55:33,565 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. after waiting 0 ms 2023-03-08 19:55:33,565 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. 2023-03-08 19:55:33,565 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=d392b5a609f9661af565e4c73dab47f8, regionState=CLOSED 2023-03-08 19:55:33,565 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678305333565"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678305333565"}]},"ts":"1678305333565"} 2023-03-08 19:55:33,568 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=91 2023-03-08 19:55:33,568 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=91, state=SUCCESS; CloseRegionProcedure d392b5a609f9661af565e4c73dab47f8, server=jenkins-hbase4.apache.org,33367,1678305218680 in 162 msec 2023-03-08 19:55:33,568 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-03-08 19:55:33,570 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. 2023-03-08 19:55:33,570 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for bf25ef5682cfde9aa90a5d6fe8229bdd: 2023-03-08 19:55:33,570 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d392b5a609f9661af565e4c73dab47f8, UNASSIGN in 166 msec 2023-03-08 19:55:33,571 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed bf25ef5682cfde9aa90a5d6fe8229bdd 2023-03-08 19:55:33,571 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 9c99479945efcf910b16288e607c83eb 2023-03-08 19:55:33,571 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 9c99479945efcf910b16288e607c83eb, disabling compactions & flushes 2023-03-08 19:55:33,571 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. 2023-03-08 19:55:33,571 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. 2023-03-08 19:55:33,571 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. after waiting 0 ms 2023-03-08 19:55:33,571 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. 2023-03-08 19:55:33,571 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=bf25ef5682cfde9aa90a5d6fe8229bdd, regionState=CLOSED 2023-03-08 19:55:33,571 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678305333571"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678305333571"}]},"ts":"1678305333571"} 2023-03-08 19:55:33,574 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=89 2023-03-08 19:55:33,574 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=89, state=SUCCESS; CloseRegionProcedure bf25ef5682cfde9aa90a5d6fe8229bdd, server=jenkins-hbase4.apache.org,33367,1678305218680 in 166 msec 2023-03-08 19:55:33,574 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bf25ef5682cfde9aa90a5d6fe8229bdd, UNASSIGN in 172 msec 2023-03-08 19:55:33,579 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-03-08 19:55:33,580 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. 2023-03-08 19:55:33,580 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 9c99479945efcf910b16288e607c83eb: 2023-03-08 19:55:33,581 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 9c99479945efcf910b16288e607c83eb 2023-03-08 19:55:33,581 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=9c99479945efcf910b16288e607c83eb, regionState=CLOSED 2023-03-08 19:55:33,581 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678305333581"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678305333581"}]},"ts":"1678305333581"} 2023-03-08 19:55:33,583 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=90 2023-03-08 19:55:33,584 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=90, state=SUCCESS; CloseRegionProcedure 9c99479945efcf910b16288e607c83eb, server=jenkins-hbase4.apache.org,33367,1678305218680 in 177 msec 2023-03-08 19:55:33,585 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=88 2023-03-08 19:55:33,585 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=9c99479945efcf910b16288e607c83eb, UNASSIGN in 182 msec 2023-03-08 19:55:33,585 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678305333585"}]},"ts":"1678305333585"} 2023-03-08 19:55:33,586 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLED in hbase:meta 2023-03-08 19:55:33,588 INFO [PEWorker-4] procedure.DisableTableProcedure(305): Set testCompactionWithHFileLink to state=DISABLED 2023-03-08 19:55:33,589 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; DisableTableProcedure table=testCompactionWithHFileLink in 192 msec 2023-03-08 19:55:33,700 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-03-08 19:55:33,701 INFO [Listener at localhost/34049] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testCompactionWithHFileLink, procId: 88 completed 2023-03-08 19:55:33,711 INFO [Listener at localhost/34049] client.HBaseAdmin(2586): Taking restore-failsafe snapshot: hbase-failsafe-snaptb-1678305319718-1678305333711 2023-03-08 19:55:33,712 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1580): Client=jenkins//172.31.14.131 snapshot request for:{ ss=hbase-failsafe-snaptb-1678305319718-1678305333711 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-08 19:55:33,712 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1678305333712 (current time:1678305333712). 2023-03-08 19:55:33,712 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-03-08 19:55:33,712 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] zookeeper.ReadOnlyZKClient(139): Connect 0x146fb750 to 127.0.0.1:56751 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 19:55:33,717 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@65e310e7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 19:55:33,718 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 19:55:33,720 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60544, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 19:55:33,720 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x146fb750 to 127.0.0.1:56751 2023-03-08 19:55:33,720 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 19:55:33,721 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-03-08 19:55:33,721 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] snapshot.SnapshotManager(659): Table is disabled, running snapshot entirely on master for { ss=hbase-failsafe-snaptb-1678305319718-1678305333711 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-08 19:55:33,723 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-08 19:55:33,724 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-08 19:55:33,724 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=95, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1678305933724 2023-03-08 19:55:33,725 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] snapshot.SnapshotManager(664): Started snapshot: { ss=hbase-failsafe-snaptb-1678305319718-1678305333711 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-08 19:55:33,725 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running DISABLED table snapshot hbase-failsafe-snaptb-1678305319718-1678305333711 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-03-08 19:55:33,725 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=hbase-failsafe-snaptb-1678305319718-1678305333711 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-03-08 19:55:33,725 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-03-08 19:55:33,825 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 19:55:33,826 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1678305319718-1678305333711 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 19:55:33,826 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1678305319718-1678305333711 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-08 19:55:33,826 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-03-08 19:55:34,026 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 19:55:34,027 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1678305319718-1678305333711 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 19:55:34,027 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1678305319718-1678305333711 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-08 19:55:34,028 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-03-08 19:55:34,131 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(97): Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1678305319718-1678305333711 table=testCompactionWithHFileLink type=DISABLED ttl=0 } 2023-03-08 19:55:34,133 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-08 19:55:34,133 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-08 19:55:34,133 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-08 19:55:34,133 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-08 19:55:34,133 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-08 19:55:34,133 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-08 19:55:34,133 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-08 19:55:34,133 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-08 19:55:34,136 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/0eb386651f534beda10396c53bbcc27e, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/85f0986f81a84e1888f18162717e0b79_SeqId_29_] hfiles 2023-03-08 19:55:34,136 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/3fca5386e0754389ae4e095454100867_SeqId_29_, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/e7667af31c2b473a97ad03da49d7dc26] hfiles 2023-03-08 19:55:34,136 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/0eb386651f534beda10396c53bbcc27e 2023-03-08 19:55:34,136 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/3fca5386e0754389ae4e095454100867_SeqId_29_ 2023-03-08 19:55:34,136 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/85f0986f81a84e1888f18162717e0b79_SeqId_29_ 2023-03-08 19:55:34,136 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/e7667af31c2b473a97ad03da49d7dc26 2023-03-08 19:55:34,136 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/89529938c0524656a66d7777082452c3_SeqId_29_, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/e7702e28b4254a2ab4f1ba997055c5a7] hfiles 2023-03-08 19:55:34,136 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/89529938c0524656a66d7777082452c3_SeqId_29_ 2023-03-08 19:55:34,136 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303080a5dec39dc3e4ef89f64b30dc2fed576, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030811e4bcff5d214acaabd0b534ff5d326f, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308b6b34034f725406e8ec26fd16680d6bd] hfiles 2023-03-08 19:55:34,137 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/e7702e28b4254a2ab4f1ba997055c5a7 2023-03-08 19:55:34,137 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/3): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303080a5dec39dc3e4ef89f64b30dc2fed576 2023-03-08 19:55:34,137 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/3b972ece89c34d3691dcfb92a6dfa7d7] hfiles 2023-03-08 19:55:34,137 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/7249207a38cc4f1bb9427f2354088baa] hfiles 2023-03-08 19:55:34,137 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/3b972ece89c34d3691dcfb92a6dfa7d7 2023-03-08 19:55:34,137 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (2/3): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030811e4bcff5d214acaabd0b534ff5d326f 2023-03-08 19:55:34,137 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/7249207a38cc4f1bb9427f2354088baa 2023-03-08 19:55:34,137 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (3/3): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308b6b34034f725406e8ec26fd16680d6bd 2023-03-08 19:55:34,137 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/edf3f0111126479b983a6c0fdea5b711] hfiles 2023-03-08 19:55:34,137 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/edf3f0111126479b983a6c0fdea5b711 2023-03-08 19:55:34,145 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308063437f66d8b4c478dc3e48cccd84c31, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023030876751d04d8044b56831dbe61c59ec030, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303089ba32d56247f467e90d3904a4b2aea47_del, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308b47202b6a5174e87a5ae0040bc3efca7, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308f529b81cd56b4bc38d7251edc066f179, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230308530634480de84dbdbb673bdb4d36dcfe, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303085a588abaab334b20b603cfd602cb2f9e, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303085c368878706a4756a425c85247df761f, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230308b5e613f2acc74fef836cd60172ea0d37, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230308f902a5c46446458782d4c156522990a4_del, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308336c8ea8d5a648e5a58dd344250b0346, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303083c55225899294a1a951e8ea7bdb2092b_del, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303083e3cf9a6a99e4f1ca67c739bc1f51618, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023030843a51fd1ba354014a3638289006916d9, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308a8d8d630c1774bd8bbc29cb46a189584] hfiles 2023-03-08 19:55:34,145 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/15): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308063437f66d8b4c478dc3e48cccd84c31 2023-03-08 19:55:34,145 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (2/15): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023030876751d04d8044b56831dbe61c59ec030 2023-03-08 19:55:34,145 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (3/15): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303089ba32d56247f467e90d3904a4b2aea47_del 2023-03-08 19:55:34,145 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (4/15): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308b47202b6a5174e87a5ae0040bc3efca7 2023-03-08 19:55:34,145 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (5/15): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230308f529b81cd56b4bc38d7251edc066f179 2023-03-08 19:55:34,145 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (6/15): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230308530634480de84dbdbb673bdb4d36dcfe 2023-03-08 19:55:34,145 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (7/15): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303085a588abaab334b20b603cfd602cb2f9e 2023-03-08 19:55:34,145 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (8/15): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303085c368878706a4756a425c85247df761f 2023-03-08 19:55:34,145 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (9/15): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230308b5e613f2acc74fef836cd60172ea0d37 2023-03-08 19:55:34,145 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (10/15): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230308f902a5c46446458782d4c156522990a4_del 2023-03-08 19:55:34,145 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (11/15): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308336c8ea8d5a648e5a58dd344250b0346 2023-03-08 19:55:34,145 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (12/15): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303083c55225899294a1a951e8ea7bdb2092b_del 2023-03-08 19:55:34,145 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (13/15): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303083e3cf9a6a99e4f1ca67c739bc1f51618 2023-03-08 19:55:34,145 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (14/15): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023030843a51fd1ba354014a3638289006916d9 2023-03-08 19:55:34,146 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (15/15): hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230308a8d8d630c1774bd8bbc29cb46a189584 2023-03-08 19:55:34,149 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(120): Marking snapshot{ ss=hbase-failsafe-snaptb-1678305319718-1678305333711 table=testCompactionWithHFileLink type=DISABLED ttl=0 } as finished. 2023-03-08 19:55:34,150 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for hbase-failsafe-snaptb-1678305319718-1678305333711 2023-03-08 19:55:34,150 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1678305319718-1678305333711 2023-03-08 19:55:34,328 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 19:55:34,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1678305319718-1678305333711 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 19:55:34,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1678305319718-1678305333711 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-08 19:55:34,329 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-03-08 19:55:34,584 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1678305319718-1678305333711 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.hbase-snapshot/hbase-failsafe-snaptb-1678305319718-1678305333711 2023-03-08 19:55:34,829 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 19:55:34,830 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1678305319718-1678305333711 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 19:55:34,830 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1678305319718-1678305333711 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-08 19:55:34,830 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-03-08 19:55:35,402 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot hbase-failsafe-snaptb-1678305319718-1678305333711 of table testCompactionWithHFileLink completed 2023-03-08 19:55:35,402 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1678305319718-1678305333711 2023-03-08 19:55:35,403 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1678305319718-1678305333711 2023-03-08 19:55:35,403 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running DISABLED table snapshot hbase-failsafe-snaptb-1678305319718-1678305333711 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1678305333725Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1678305319718-1678305333711 table=testCompactionWithHFileLink type=DISABLED ttl=0 } at 1678305334131 (+406 ms)Consolidate snapshot: hbase-failsafe-snaptb-1678305319718-1678305333711 at 1678305334150 (+19 ms)Loading Region manifests for hbase-failsafe-snaptb-1678305319718-1678305333711 at 1678305334150Writing data manifest for hbase-failsafe-snaptb-1678305319718-1678305333711 at 1678305334152 (+2 ms)Verifying snapshot: hbase-failsafe-snaptb-1678305319718-1678305333711 at 1678305334558 (+406 ms)Snapshot hbase-failsafe-snaptb-1678305319718-1678305333711 of table testCompactionWithHFileLink completed at 1678305335402 (+844 ms) 2023-03-08 19:55:35,404 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=95, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-08 19:55:35,405 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=95, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 1.6810 sec 2023-03-08 19:55:35,831 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-08 19:55:35,832 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1678305319718-1678305333711 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-08 19:55:35,832 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] snapshot.SnapshotManager(401): Snapshot '{ ss=hbase-failsafe-snaptb-1678305319718-1678305333711 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-03-08 19:55:35,875 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:RESTORE_SNAPSHOT_PRE_OPERATION; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1678305319718" table: "testCompactionWithHFileLink" creation_time: 1678305319728 type: FLUSH version: 2 ttl: 0 ) 2023-03-08 19:55:35,875 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] snapshot.SnapshotManager(880): Restore snapshot=snaptb-1678305319718 as table=testCompactionWithHFileLink 2023-03-08 19:55:35,878 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-08 19:55:35,889 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-03-08 19:55:35,890 DEBUG [PEWorker-3] util.FSTableDescriptors(519): Deleted hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-03-08 19:55:35,890 INFO [PEWorker-3] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-03-08 19:55:35,891 INFO [PEWorker-3] procedure.RestoreSnapshotProcedure(374): Starting restore snapshot={ ss=snaptb-1678305319718 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-08 19:55:35,896 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(177): starting restore table regions using snapshot=name: "snaptb-1678305319718" table: "testCompactionWithHFileLink" creation_time: 1678305319728 type: FLUSH version: 2 ttl: 0 2023-03-08 19:55:35,897 DEBUG [PEWorker-3] snapshot.RestoreSnapshotHelper(785): get table regions: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink 2023-03-08 19:55:35,902 DEBUG [PEWorker-3] snapshot.RestoreSnapshotHelper(797): found 3 regions for table=testCompactionWithHFileLink 2023-03-08 19:55:35,905 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303080a5dec39dc3e4ef89f64b30dc2fed576 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303080a5dec39dc3e4ef89f64b30dc2fed576 2023-03-08 19:55:35,907 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030811e4bcff5d214acaabd0b534ff5d326f to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030811e4bcff5d214acaabd0b534ff5d326f 2023-03-08 19:55:35,908 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308b6b34034f725406e8ec26fd16680d6bd to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308b6b34034f725406e8ec26fd16680d6bd 2023-03-08 19:55:35,908 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202303083653e76c42154508bdba69de9d712f83 of snapshot=snaptb-1678305319718 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 19:55:35,911 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202303088ee4da827de14300899601e052c73946 of snapshot=snaptb-1678305319718 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 19:55:35,915 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230308d3c2ea31978640cf9022c0e5fb535d1c of snapshot=snaptb-1678305319718 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 19:55:35,918 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230308e7a0a9d515b24f63a4013e970fb178a0 of snapshot=snaptb-1678305319718 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 19:55:35,921 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c2023030803a51c78c3a9414481e4489a8fc504ad of snapshot=snaptb-1678305319718 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 19:55:35,924 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202303088ffe3f5d37194c0fb226cac523d21fef of snapshot=snaptb-1678305319718 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 19:55:35,928 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20230308c5433010db254bcfb902142789f105cc of snapshot=snaptb-1678305319718 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 19:55:35,931 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20230308dd1c4643f5654221b89fb3ab8d9a8eb4 of snapshot=snaptb-1678305319718 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 19:55:35,934 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e202303088d68f70559bb4783aaa77c40b3a3e2c0 of snapshot=snaptb-1678305319718 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 19:55:35,938 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e202303089f52416d158d4f62ac1989a0c5c86166 of snapshot=snaptb-1678305319718 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 19:55:35,948 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230308b87a29e85e7140f29524a1dd29216255 of snapshot=snaptb-1678305319718 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 19:55:35,951 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230308c0451f6624404cb9a261ce5d697fda74 of snapshot=snaptb-1678305319718 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-08 19:55:35,958 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230308f902a5c46446458782d4c156522990a4_del to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230308f902a5c46446458782d4c156522990a4_del 2023-03-08 19:55:35,959 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303083c55225899294a1a951e8ea7bdb2092b_del to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303083c55225899294a1a951e8ea7bdb2092b_del 2023-03-08 19:55:35,960 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303089ba32d56247f467e90d3904a4b2aea47_del to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303089ba32d56247f467e90d3904a4b2aea47_del 2023-03-08 19:55:35,960 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(223): region to restore: 9c99479945efcf910b16288e607c83eb 2023-03-08 19:55:35,961 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(223): region to restore: bf25ef5682cfde9aa90a5d6fe8229bdd 2023-03-08 19:55:35,961 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(223): region to restore: d392b5a609f9661af565e4c73dab47f8 2023-03-08 19:55:35,966 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/89529938c0524656a66d7777082452c3_SeqId_29_ to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/89529938c0524656a66d7777082452c3_SeqId_29_ 2023-03-08 19:55:35,966 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/85f0986f81a84e1888f18162717e0b79_SeqId_29_ to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/85f0986f81a84e1888f18162717e0b79_SeqId_29_ 2023-03-08 19:55:35,966 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/3fca5386e0754389ae4e095454100867_SeqId_29_ to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/3fca5386e0754389ae4e095454100867_SeqId_29_ 2023-03-08 19:55:35,968 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/e7702e28b4254a2ab4f1ba997055c5a7 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/e7702e28b4254a2ab4f1ba997055c5a7 2023-03-08 19:55:35,968 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 8cb2d3dd62cc42009c2420fabc400fb6 of snapshot=snaptb-1678305319718 to region=bf25ef5682cfde9aa90a5d6fe8229bdd table=testCompactionWithHFileLink 2023-03-08 19:55:35,968 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/e7667af31c2b473a97ad03da49d7dc26 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/e7667af31c2b473a97ad03da49d7dc26 2023-03-08 19:55:35,968 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 123a27e9c7024e02b0fbca054fd86280 of snapshot=snaptb-1678305319718 to region=9c99479945efcf910b16288e607c83eb table=testCompactionWithHFileLink 2023-03-08 19:55:35,971 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/0eb386651f534beda10396c53bbcc27e to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/0eb386651f534beda10396c53bbcc27e 2023-03-08 19:55:35,971 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 7372b15acde5432d9949bd6d62365bd8 of snapshot=snaptb-1678305319718 to region=d392b5a609f9661af565e4c73dab47f8 table=testCompactionWithHFileLink 2023-03-08 19:55:35,974 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink a433c1e1bdb446f385d1156aa8209307 of snapshot=snaptb-1678305319718 to region=9c99479945efcf910b16288e607c83eb table=testCompactionWithHFileLink 2023-03-08 19:55:35,976 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink e39643d9b2424962b7f3883e1e919f9a of snapshot=snaptb-1678305319718 to region=d392b5a609f9661af565e4c73dab47f8 table=testCompactionWithHFileLink 2023-03-08 19:55:35,978 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 9a28a7cd1d794cde8d25ec94745b3bb9 of snapshot=snaptb-1678305319718 to region=9c99479945efcf910b16288e607c83eb table=testCompactionWithHFileLink 2023-03-08 19:55:35,978 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink af1975c5b195440797e41fe2eba32cc4 of snapshot=snaptb-1678305319718 to region=bf25ef5682cfde9aa90a5d6fe8229bdd table=testCompactionWithHFileLink 2023-03-08 19:55:35,980 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-08 19:55:35,980 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c285d649295f485d986c468697edc0ce of snapshot=snaptb-1678305319718 to region=d392b5a609f9661af565e4c73dab47f8 table=testCompactionWithHFileLink 2023-03-08 19:55:35,982 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 046ec2868b184c3e84b6105bf189d3bd of snapshot=snaptb-1678305319718 to region=bf25ef5682cfde9aa90a5d6fe8229bdd table=testCompactionWithHFileLink 2023-03-08 19:55:35,983 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink f7d2cb4000d943219c0ae3c05e060127 of snapshot=snaptb-1678305319718 to region=9c99479945efcf910b16288e607c83eb table=testCompactionWithHFileLink 2023-03-08 19:55:35,984 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 67ebbe2e123e43d18dd53054062690ec of snapshot=snaptb-1678305319718 to region=d392b5a609f9661af565e4c73dab47f8 table=testCompactionWithHFileLink 2023-03-08 19:55:35,989 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c9ebe20caf5d4c2cb57adb8d0820d554 of snapshot=snaptb-1678305319718 to region=bf25ef5682cfde9aa90a5d6fe8229bdd table=testCompactionWithHFileLink 2023-03-08 19:55:35,992 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/7249207a38cc4f1bb9427f2354088baa to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/7249207a38cc4f1bb9427f2354088baa 2023-03-08 19:55:35,992 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 07773024718d41078416d0bd2e154765 of snapshot=snaptb-1678305319718 to region=d392b5a609f9661af565e4c73dab47f8 table=testCompactionWithHFileLink 2023-03-08 19:55:35,992 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/3b972ece89c34d3691dcfb92a6dfa7d7 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/3b972ece89c34d3691dcfb92a6dfa7d7 2023-03-08 19:55:35,992 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink cdaeaeb4819a41cc88f69b9970a33dcb of snapshot=snaptb-1678305319718 to region=9c99479945efcf910b16288e607c83eb table=testCompactionWithHFileLink 2023-03-08 19:55:35,996 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 3e46bffcb555430ca182ded732797e54 of snapshot=snaptb-1678305319718 to region=d392b5a609f9661af565e4c73dab47f8 table=testCompactionWithHFileLink 2023-03-08 19:55:35,996 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/edf3f0111126479b983a6c0fdea5b711 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/edf3f0111126479b983a6c0fdea5b711 2023-03-08 19:55:35,996 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 044f1e6d183045beae5b75645bb97f60 of snapshot=snaptb-1678305319718 to region=bf25ef5682cfde9aa90a5d6fe8229bdd table=testCompactionWithHFileLink 2023-03-08 19:55:35,996 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 68353db2020f4adeba9aeb341538de89 of snapshot=snaptb-1678305319718 to region=9c99479945efcf910b16288e607c83eb table=testCompactionWithHFileLink 2023-03-08 19:55:36,000 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 6eeb48f439a34bc9ae7493396a8ad303 of snapshot=snaptb-1678305319718 to region=d392b5a609f9661af565e4c73dab47f8 table=testCompactionWithHFileLink 2023-03-08 19:55:36,000 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 7957d21df4a84e90b22385e13d00adde of snapshot=snaptb-1678305319718 to region=9c99479945efcf910b16288e607c83eb table=testCompactionWithHFileLink 2023-03-08 19:55:36,003 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d9b3dfdcafcf4aa3b87dfdc6c038bf9e of snapshot=snaptb-1678305319718 to region=bf25ef5682cfde9aa90a5d6fe8229bdd table=testCompactionWithHFileLink 2023-03-08 19:55:36,005 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 3f39c3b5bff246788da3275e205043fb of snapshot=snaptb-1678305319718 to region=9c99479945efcf910b16288e607c83eb table=testCompactionWithHFileLink 2023-03-08 19:55:36,005 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink a84f2cc20b89462cab9556cca240ba20 of snapshot=snaptb-1678305319718 to region=d392b5a609f9661af565e4c73dab47f8 table=testCompactionWithHFileLink 2023-03-08 19:55:36,007 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d97fbd02a2e04d4cb97a3bfe7772fb6e of snapshot=snaptb-1678305319718 to region=bf25ef5682cfde9aa90a5d6fe8229bdd table=testCompactionWithHFileLink 2023-03-08 19:55:36,015 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink e323979e94844e49a28882288ab3a208 of snapshot=snaptb-1678305319718 to region=bf25ef5682cfde9aa90a5d6fe8229bdd table=testCompactionWithHFileLink 2023-03-08 19:55:36,018 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(266): finishing restore table regions using snapshot=name: "snaptb-1678305319718" table: "testCompactionWithHFileLink" creation_time: 1678305319728 type: FLUSH version: 2 ttl: 0 2023-03-08 19:55:36,019 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1678305336019"}]},"ts":"9223372036854775807"} 2023-03-08 19:55:36,019 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1678305336019"}]},"ts":"9223372036854775807"} 2023-03-08 19:55:36,019 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1678305336019"}]},"ts":"9223372036854775807"} 2023-03-08 19:55:36,021 INFO [PEWorker-3] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-03-08 19:55:36,021 DEBUG [PEWorker-3] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 9c99479945efcf910b16288e607c83eb, NAME => 'testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb.', STARTKEY => '1', ENDKEY => '2'}, {ENCODED => bf25ef5682cfde9aa90a5d6fe8229bdd, NAME => 'testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => d392b5a609f9661af565e4c73dab47f8, NAME => 'testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8.', STARTKEY => '2', ENDKEY => ''}] 2023-03-08 19:55:36,021 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678305336020"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678305336020"}]},"ts":"1678305336020"} 2023-03-08 19:55:36,021 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678305336020"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678305336020"}]},"ts":"1678305336020"} 2023-03-08 19:55:36,021 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678305336020"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678305336020"}]},"ts":"1678305336020"} 2023-03-08 19:55:36,023 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-08 19:55:36,023 INFO [PEWorker-3] hbase.MetaTableAccessor(1810): Overwritten 3 regions to Meta 2023-03-08 19:55:36,023 DEBUG [PEWorker-3] hbase.MetaTableAccessor(1811): Overwritten regions: [{ENCODED => 9c99479945efcf910b16288e607c83eb, NAME => 'testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb.', STARTKEY => '1', ENDKEY => '2'}, {ENCODED => bf25ef5682cfde9aa90a5d6fe8229bdd, NAME => 'testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => d392b5a609f9661af565e4c73dab47f8, NAME => 'testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8.', STARTKEY => '2', ENDKEY => ''}] 2023-03-08 19:55:36,023 INFO [PEWorker-3] procedure.RestoreSnapshotProcedure(447): Restore snapshot={ ss=snaptb-1678305319718 table=testCompactionWithHFileLink type=FLUSH ttl=0 } on table=testCompactionWithHFileLink completed! 2023-03-08 19:55:36,025 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1678305319718" table: "testCompactionWithHFileLink" creation_time: 1678305319728 type: FLUSH version: 2 ttl: 0 ) in 160 msec 2023-03-08 19:55:36,181 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-08 19:55:36,181 INFO [Listener at localhost/34049] client.HBaseAdmin$TableFuture(3541): Operation: MODIFY, Table Name: default:testCompactionWithHFileLink, procId: 96 completed 2023-03-08 19:55:36,181 INFO [Listener at localhost/34049] client.HBaseAdmin(2618): Deleting restore-failsafe snapshot: hbase-failsafe-snaptb-1678305319718-1678305333711 2023-03-08 19:55:36,187 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(748): Client=jenkins//172.31.14.131 delete name: "hbase-failsafe-snaptb-1678305319718-1678305333711" 2023-03-08 19:55:36,189 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] snapshot.SnapshotManager(333): Deleting snapshot: hbase-failsafe-snaptb-1678305319718-1678305333711 2023-03-08 19:55:36,192 INFO [Listener at localhost/34049] client.HBaseAdmin$14(830): Started enable of testCompactionWithHFileLink 2023-03-08 19:55:36,197 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.HMaster$10(2385): Client=jenkins//172.31.14.131 enable testCompactionWithHFileLink 2023-03-08 19:55:36,203 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure2.ProcedureExecutor(1029): Stored pid=97, state=RUNNABLE:ENABLE_TABLE_PREPARE; EnableTableProcedure table=testCompactionWithHFileLink 2023-03-08 19:55:36,205 INFO [PEWorker-5] procedure.EnableTableProcedure(296): Attempting to enable the table testCompactionWithHFileLink 2023-03-08 19:55:36,206 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678305336206"}]},"ts":"1678305336206"} 2023-03-08 19:55:36,206 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-08 19:55:36,207 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-03-08 19:55:36,209 DEBUG [PEWorker-5] procedure.EnableTableProcedure(107): No change in number of region replicas (configuredReplicaCount=1); assigning. 2023-03-08 19:55:36,209 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bf25ef5682cfde9aa90a5d6fe8229bdd, ASSIGN}, {pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=9c99479945efcf910b16288e607c83eb, ASSIGN}, {pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d392b5a609f9661af565e4c73dab47f8, ASSIGN}] 2023-03-08 19:55:36,210 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=9c99479945efcf910b16288e607c83eb, ASSIGN 2023-03-08 19:55:36,210 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bf25ef5682cfde9aa90a5d6fe8229bdd, ASSIGN 2023-03-08 19:55:36,210 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d392b5a609f9661af565e4c73dab47f8, ASSIGN 2023-03-08 19:55:36,211 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bf25ef5682cfde9aa90a5d6fe8229bdd, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-03-08 19:55:36,211 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d392b5a609f9661af565e4c73dab47f8, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-03-08 19:55:36,211 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=9c99479945efcf910b16288e607c83eb, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-03-08 19:55:36,307 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-08 19:55:36,362 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=9c99479945efcf910b16288e607c83eb, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:36,362 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=d392b5a609f9661af565e4c73dab47f8, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:36,362 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=bf25ef5682cfde9aa90a5d6fe8229bdd, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:36,363 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678305336362"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678305336362"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678305336362"}]},"ts":"1678305336362"} 2023-03-08 19:55:36,363 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678305336362"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678305336362"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678305336362"}]},"ts":"1678305336362"} 2023-03-08 19:55:36,363 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678305336362"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678305336362"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678305336362"}]},"ts":"1678305336362"} 2023-03-08 19:55:36,365 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=101, ppid=99, state=RUNNABLE; OpenRegionProcedure 9c99479945efcf910b16288e607c83eb, server=jenkins-hbase4.apache.org,33367,1678305218680}] 2023-03-08 19:55:36,365 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=102, ppid=100, state=RUNNABLE; OpenRegionProcedure d392b5a609f9661af565e4c73dab47f8, server=jenkins-hbase4.apache.org,33367,1678305218680}] 2023-03-08 19:55:36,366 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=98, state=RUNNABLE; OpenRegionProcedure bf25ef5682cfde9aa90a5d6fe8229bdd, server=jenkins-hbase4.apache.org,33367,1678305218680}] 2023-03-08 19:55:36,508 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-08 19:55:36,518 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. 2023-03-08 19:55:36,518 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d392b5a609f9661af565e4c73dab47f8, NAME => 'testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8.', STARTKEY => '2', ENDKEY => ''} 2023-03-08 19:55:36,518 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink d392b5a609f9661af565e4c73dab47f8 2023-03-08 19:55:36,519 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:55:36,519 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d392b5a609f9661af565e4c73dab47f8 2023-03-08 19:55:36,519 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d392b5a609f9661af565e4c73dab47f8 2023-03-08 19:55:36,520 INFO [StoreOpener-d392b5a609f9661af565e4c73dab47f8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region d392b5a609f9661af565e4c73dab47f8 2023-03-08 19:55:36,521 DEBUG [StoreOpener-d392b5a609f9661af565e4c73dab47f8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1 2023-03-08 19:55:36,521 DEBUG [StoreOpener-d392b5a609f9661af565e4c73dab47f8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1 2023-03-08 19:55:36,521 INFO [StoreOpener-d392b5a609f9661af565e4c73dab47f8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d392b5a609f9661af565e4c73dab47f8 columnFamilyName family1 2023-03-08 19:55:36,531 DEBUG [StoreOpener-d392b5a609f9661af565e4c73dab47f8-1] regionserver.HStore(539): loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-67ebbe2e123e43d18dd53054062690ec 2023-03-08 19:55:36,537 DEBUG [StoreOpener-d392b5a609f9661af565e4c73dab47f8-1] regionserver.HStore(539): loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-7372b15acde5432d9949bd6d62365bd8 2023-03-08 19:55:36,542 DEBUG [StoreOpener-d392b5a609f9661af565e4c73dab47f8-1] regionserver.HStore(539): loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-c285d649295f485d986c468697edc0ce 2023-03-08 19:55:36,547 DEBUG [StoreOpener-d392b5a609f9661af565e4c73dab47f8-1] regionserver.HStore(539): loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-e39643d9b2424962b7f3883e1e919f9a 2023-03-08 19:55:36,548 INFO [StoreOpener-d392b5a609f9661af565e4c73dab47f8-1] regionserver.HStore(310): Store=d392b5a609f9661af565e4c73dab47f8/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:55:36,548 INFO [StoreOpener-d392b5a609f9661af565e4c73dab47f8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region d392b5a609f9661af565e4c73dab47f8 2023-03-08 19:55:36,549 DEBUG [StoreOpener-d392b5a609f9661af565e4c73dab47f8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2 2023-03-08 19:55:36,549 DEBUG [StoreOpener-d392b5a609f9661af565e4c73dab47f8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2 2023-03-08 19:55:36,550 INFO [StoreOpener-d392b5a609f9661af565e4c73dab47f8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d392b5a609f9661af565e4c73dab47f8 columnFamilyName family2 2023-03-08 19:55:36,557 DEBUG [StoreOpener-d392b5a609f9661af565e4c73dab47f8-1] regionserver.HStore(539): loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-07773024718d41078416d0bd2e154765 2023-03-08 19:55:36,562 DEBUG [StoreOpener-d392b5a609f9661af565e4c73dab47f8-1] regionserver.HStore(539): loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-3e46bffcb555430ca182ded732797e54 2023-03-08 19:55:36,567 DEBUG [StoreOpener-d392b5a609f9661af565e4c73dab47f8-1] regionserver.HStore(539): loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-6eeb48f439a34bc9ae7493396a8ad303 2023-03-08 19:55:36,572 DEBUG [StoreOpener-d392b5a609f9661af565e4c73dab47f8-1] regionserver.HStore(539): loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-a84f2cc20b89462cab9556cca240ba20 2023-03-08 19:55:36,572 INFO [StoreOpener-d392b5a609f9661af565e4c73dab47f8-1] regionserver.HStore(310): Store=d392b5a609f9661af565e4c73dab47f8/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:55:36,573 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8 2023-03-08 19:55:36,574 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8 2023-03-08 19:55:36,576 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 19:55:36,577 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d392b5a609f9661af565e4c73dab47f8 2023-03-08 19:55:36,578 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d392b5a609f9661af565e4c73dab47f8; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=58807720, jitterRate=-0.12369668483734131}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 19:55:36,578 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d392b5a609f9661af565e4c73dab47f8: 2023-03-08 19:55:36,579 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8., pid=102, masterSystemTime=1678305336516 2023-03-08 19:55:36,579 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 19:55:36,580 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-08 19:55:36,580 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-03-08 19:55:36,580 DEBUG [RS:0;jenkins-hbase4:33367-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-08 19:55:36,581 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. 2023-03-08 19:55:36,582 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. 2023-03-08 19:55:36,582 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. 2023-03-08 19:55:36,582 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => bf25ef5682cfde9aa90a5d6fe8229bdd, NAME => 'testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd.', STARTKEY => '', ENDKEY => '1'} 2023-03-08 19:55:36,582 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink bf25ef5682cfde9aa90a5d6fe8229bdd 2023-03-08 19:55:36,582 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:55:36,582 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for bf25ef5682cfde9aa90a5d6fe8229bdd 2023-03-08 19:55:36,582 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for bf25ef5682cfde9aa90a5d6fe8229bdd 2023-03-08 19:55:36,583 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=d392b5a609f9661af565e4c73dab47f8, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:36,583 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678305336583"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678305336583"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678305336583"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678305336583"}]},"ts":"1678305336583"} 2023-03-08 19:55:36,584 INFO [StoreOpener-bf25ef5682cfde9aa90a5d6fe8229bdd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region bf25ef5682cfde9aa90a5d6fe8229bdd 2023-03-08 19:55:36,586 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-08 19:55:36,586 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1912): d392b5a609f9661af565e4c73dab47f8/family1 is initiating minor compaction (all files) 2023-03-08 19:55:36,586 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of d392b5a609f9661af565e4c73dab47f8/family1 in testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. 2023-03-08 19:55:36,586 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-7372b15acde5432d9949bd6d62365bd8, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-e39643d9b2424962b7f3883e1e919f9a, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-c285d649295f485d986c468697edc0ce, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-67ebbe2e123e43d18dd53054062690ec] into tmpdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/.tmp, totalSize=21.6 K 2023-03-08 19:55:36,587 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=102, resume processing ppid=100 2023-03-08 19:55:36,587 DEBUG [RS:0;jenkins-hbase4:33367-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-08 19:55:36,587 DEBUG [StoreOpener-bf25ef5682cfde9aa90a5d6fe8229bdd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1 2023-03-08 19:55:36,587 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=102, ppid=100, state=SUCCESS; OpenRegionProcedure d392b5a609f9661af565e4c73dab47f8, server=jenkins-hbase4.apache.org,33367,1678305218680 in 220 msec 2023-03-08 19:55:36,587 DEBUG [StoreOpener-bf25ef5682cfde9aa90a5d6fe8229bdd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1 2023-03-08 19:55:36,587 DEBUG [RS:0;jenkins-hbase4:33367-longCompactions-0] regionserver.HStore(1912): d392b5a609f9661af565e4c73dab47f8/family2 is initiating minor compaction (all files) 2023-03-08 19:55:36,587 INFO [RS:0;jenkins-hbase4:33367-longCompactions-0] regionserver.HRegion(2259): Starting compaction of d392b5a609f9661af565e4c73dab47f8/family2 in testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. 2023-03-08 19:55:36,587 INFO [RS:0;jenkins-hbase4:33367-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-07773024718d41078416d0bd2e154765, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-3e46bffcb555430ca182ded732797e54, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-6eeb48f439a34bc9ae7493396a8ad303, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-a84f2cc20b89462cab9556cca240ba20] into tmpdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/.tmp, totalSize=20.4 K 2023-03-08 19:55:36,588 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-7372b15acde5432d9949bd6d62365bd8, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678305313718 2023-03-08 19:55:36,588 INFO [StoreOpener-bf25ef5682cfde9aa90a5d6fe8229bdd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bf25ef5682cfde9aa90a5d6fe8229bdd columnFamilyName family1 2023-03-08 19:55:36,588 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d392b5a609f9661af565e4c73dab47f8, ASSIGN in 378 msec 2023-03-08 19:55:36,589 DEBUG [RS:0;jenkins-hbase4:33367-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-07773024718d41078416d0bd2e154765, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678305313718 2023-03-08 19:55:36,589 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-e39643d9b2424962b7f3883e1e919f9a, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678305315116 2023-03-08 19:55:36,589 DEBUG [RS:0;jenkins-hbase4:33367-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-3e46bffcb555430ca182ded732797e54, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678305315116 2023-03-08 19:55:36,590 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-c285d649295f485d986c468697edc0ce, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678305316519 2023-03-08 19:55:36,591 DEBUG [RS:0;jenkins-hbase4:33367-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-6eeb48f439a34bc9ae7493396a8ad303, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678305316519 2023-03-08 19:55:36,591 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-67ebbe2e123e43d18dd53054062690ec, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678305317923 2023-03-08 19:55:36,592 DEBUG [RS:0;jenkins-hbase4:33367-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-a84f2cc20b89462cab9556cca240ba20, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678305317923 2023-03-08 19:55:36,603 DEBUG [StoreOpener-bf25ef5682cfde9aa90a5d6fe8229bdd-1] regionserver.HStore(539): loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-046ec2868b184c3e84b6105bf189d3bd 2023-03-08 19:55:36,610 DEBUG [StoreOpener-bf25ef5682cfde9aa90a5d6fe8229bdd-1] regionserver.HStore(539): loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-8cb2d3dd62cc42009c2420fabc400fb6 2023-03-08 19:55:36,616 DEBUG [StoreOpener-bf25ef5682cfde9aa90a5d6fe8229bdd-1] regionserver.HStore(539): loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-af1975c5b195440797e41fe2eba32cc4 2023-03-08 19:55:36,620 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] throttle.PressureAwareThroughputController(145): d392b5a609f9661af565e4c73dab47f8#family1#compaction#114 average throughput is 0.78 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-03-08 19:55:36,620 INFO [RS:0;jenkins-hbase4:33367-longCompactions-0] throttle.PressureAwareThroughputController(145): d392b5a609f9661af565e4c73dab47f8#family2#compaction#115 average throughput is 0.39 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 19:55:36,624 DEBUG [StoreOpener-bf25ef5682cfde9aa90a5d6fe8229bdd-1] regionserver.HStore(539): loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-c9ebe20caf5d4c2cb57adb8d0820d554 2023-03-08 19:55:36,624 INFO [StoreOpener-bf25ef5682cfde9aa90a5d6fe8229bdd-1] regionserver.HStore(310): Store=bf25ef5682cfde9aa90a5d6fe8229bdd/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:55:36,624 INFO [StoreOpener-bf25ef5682cfde9aa90a5d6fe8229bdd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region bf25ef5682cfde9aa90a5d6fe8229bdd 2023-03-08 19:55:36,625 DEBUG [StoreOpener-bf25ef5682cfde9aa90a5d6fe8229bdd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2 2023-03-08 19:55:36,626 DEBUG [StoreOpener-bf25ef5682cfde9aa90a5d6fe8229bdd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2 2023-03-08 19:55:36,626 INFO [StoreOpener-bf25ef5682cfde9aa90a5d6fe8229bdd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bf25ef5682cfde9aa90a5d6fe8229bdd columnFamilyName family2 2023-03-08 19:55:36,638 DEBUG [StoreOpener-bf25ef5682cfde9aa90a5d6fe8229bdd-1] regionserver.HStore(539): loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-044f1e6d183045beae5b75645bb97f60 2023-03-08 19:55:36,646 DEBUG [StoreOpener-bf25ef5682cfde9aa90a5d6fe8229bdd-1] regionserver.HStore(539): loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-d97fbd02a2e04d4cb97a3bfe7772fb6e 2023-03-08 19:55:36,654 DEBUG [StoreOpener-bf25ef5682cfde9aa90a5d6fe8229bdd-1] regionserver.HStore(539): loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-d9b3dfdcafcf4aa3b87dfdc6c038bf9e 2023-03-08 19:55:36,662 DEBUG [StoreOpener-bf25ef5682cfde9aa90a5d6fe8229bdd-1] regionserver.HStore(539): loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-e323979e94844e49a28882288ab3a208 2023-03-08 19:55:36,662 INFO [StoreOpener-bf25ef5682cfde9aa90a5d6fe8229bdd-1] regionserver.HStore(310): Store=bf25ef5682cfde9aa90a5d6fe8229bdd/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:55:36,663 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd 2023-03-08 19:55:36,664 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd 2023-03-08 19:55:36,665 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 19:55:36,666 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for bf25ef5682cfde9aa90a5d6fe8229bdd 2023-03-08 19:55:36,666 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened bf25ef5682cfde9aa90a5d6fe8229bdd; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70612258, jitterRate=0.052204638719558716}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 19:55:36,666 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for bf25ef5682cfde9aa90a5d6fe8229bdd: 2023-03-08 19:55:36,667 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd., pid=103, masterSystemTime=1678305336516 2023-03-08 19:55:36,667 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-03-08 19:55:36,667 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-03-08 19:55:36,669 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. 2023-03-08 19:55:36,669 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. 2023-03-08 19:55:36,669 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. 2023-03-08 19:55:36,669 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 9c99479945efcf910b16288e607c83eb, NAME => 'testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb.', STARTKEY => '1', ENDKEY => '2'} 2023-03-08 19:55:36,669 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=bf25ef5682cfde9aa90a5d6fe8229bdd, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:36,669 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 9c99479945efcf910b16288e607c83eb 2023-03-08 19:55:36,669 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:55:36,669 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678305336669"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678305336669"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678305336669"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678305336669"}]},"ts":"1678305336669"} 2023-03-08 19:55:36,669 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 9c99479945efcf910b16288e607c83eb 2023-03-08 19:55:36,669 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 9c99479945efcf910b16288e607c83eb 2023-03-08 19:55:36,671 INFO [StoreOpener-9c99479945efcf910b16288e607c83eb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 9c99479945efcf910b16288e607c83eb 2023-03-08 19:55:36,671 DEBUG [StoreOpener-9c99479945efcf910b16288e607c83eb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1 2023-03-08 19:55:36,672 DEBUG [StoreOpener-9c99479945efcf910b16288e607c83eb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1 2023-03-08 19:55:36,672 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=98 2023-03-08 19:55:36,672 INFO [StoreOpener-9c99479945efcf910b16288e607c83eb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9c99479945efcf910b16288e607c83eb columnFamilyName family1 2023-03-08 19:55:36,672 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=98, state=SUCCESS; OpenRegionProcedure bf25ef5682cfde9aa90a5d6fe8229bdd, server=jenkins-hbase4.apache.org,33367,1678305218680 in 305 msec 2023-03-08 19:55:36,673 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bf25ef5682cfde9aa90a5d6fe8229bdd, ASSIGN in 463 msec 2023-03-08 19:55:36,678 DEBUG [StoreOpener-9c99479945efcf910b16288e607c83eb-1] regionserver.HStore(539): loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-123a27e9c7024e02b0fbca054fd86280 2023-03-08 19:55:36,685 DEBUG [StoreOpener-9c99479945efcf910b16288e607c83eb-1] regionserver.HStore(539): loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-9a28a7cd1d794cde8d25ec94745b3bb9 2023-03-08 19:55:36,692 DEBUG [StoreOpener-9c99479945efcf910b16288e607c83eb-1] regionserver.HStore(539): loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-a433c1e1bdb446f385d1156aa8209307 2023-03-08 19:55:36,699 DEBUG [StoreOpener-9c99479945efcf910b16288e607c83eb-1] regionserver.HStore(539): loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-f7d2cb4000d943219c0ae3c05e060127 2023-03-08 19:55:36,700 INFO [StoreOpener-9c99479945efcf910b16288e607c83eb-1] regionserver.HStore(310): Store=9c99479945efcf910b16288e607c83eb/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:55:36,700 INFO [StoreOpener-9c99479945efcf910b16288e607c83eb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 9c99479945efcf910b16288e607c83eb 2023-03-08 19:55:36,701 DEBUG [StoreOpener-9c99479945efcf910b16288e607c83eb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2 2023-03-08 19:55:36,701 DEBUG [StoreOpener-9c99479945efcf910b16288e607c83eb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2 2023-03-08 19:55:36,701 INFO [StoreOpener-9c99479945efcf910b16288e607c83eb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9c99479945efcf910b16288e607c83eb columnFamilyName family2 2023-03-08 19:55:36,711 DEBUG [StoreOpener-9c99479945efcf910b16288e607c83eb-1] regionserver.HStore(539): loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-3f39c3b5bff246788da3275e205043fb 2023-03-08 19:55:36,718 DEBUG [StoreOpener-9c99479945efcf910b16288e607c83eb-1] regionserver.HStore(539): loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-68353db2020f4adeba9aeb341538de89 2023-03-08 19:55:36,731 DEBUG [StoreOpener-9c99479945efcf910b16288e607c83eb-1] regionserver.HStore(539): loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-7957d21df4a84e90b22385e13d00adde 2023-03-08 19:55:36,741 DEBUG [StoreOpener-9c99479945efcf910b16288e607c83eb-1] regionserver.HStore(539): loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-cdaeaeb4819a41cc88f69b9970a33dcb 2023-03-08 19:55:36,741 INFO [StoreOpener-9c99479945efcf910b16288e607c83eb-1] regionserver.HStore(310): Store=9c99479945efcf910b16288e607c83eb/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:55:36,742 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb 2023-03-08 19:55:36,743 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb 2023-03-08 19:55:36,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 19:55:36,748 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 9c99479945efcf910b16288e607c83eb 2023-03-08 19:55:36,749 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 9c99479945efcf910b16288e607c83eb; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=58731167, jitterRate=-0.12483741343021393}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 19:55:36,749 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 9c99479945efcf910b16288e607c83eb: 2023-03-08 19:55:36,750 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb., pid=101, masterSystemTime=1678305336516 2023-03-08 19:55:36,750 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-03-08 19:55:36,750 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-03-08 19:55:36,751 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. 2023-03-08 19:55:36,751 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. 2023-03-08 19:55:36,751 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=9c99479945efcf910b16288e607c83eb, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:36,752 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678305336751"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678305336751"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678305336751"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678305336751"}]},"ts":"1678305336751"} 2023-03-08 19:55:36,756 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=101, resume processing ppid=99 2023-03-08 19:55:36,756 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=101, ppid=99, state=SUCCESS; OpenRegionProcedure 9c99479945efcf910b16288e607c83eb, server=jenkins-hbase4.apache.org,33367,1678305218680 in 388 msec 2023-03-08 19:55:36,757 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=97 2023-03-08 19:55:36,757 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=9c99479945efcf910b16288e607c83eb, ASSIGN in 547 msec 2023-03-08 19:55:36,758 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678305336758"}]},"ts":"1678305336758"} 2023-03-08 19:55:36,759 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-03-08 19:55:36,761 INFO [PEWorker-2] procedure.EnableTableProcedure(310): Table 'testCompactionWithHFileLink' was successfully enabled. 2023-03-08 19:55:36,762 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=97, state=SUCCESS; EnableTableProcedure table=testCompactionWithHFileLink in 563 msec 2023-03-08 19:55:36,809 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-08 19:55:36,809 INFO [Listener at localhost/34049] client.HBaseAdmin$TableFuture(3541): Operation: ENABLE, Table Name: default:testCompactionWithHFileLink, procId: 97 completed 2023-03-08 19:55:36,824 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 19:55:36,832 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 19:55:36,840 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 19:55:36,841 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 19:55:36,847 INFO [Listener at localhost/34049] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-08 19:55:36,860 INFO [Listener at localhost/34049] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-03-08 19:55:36,860 INFO [Listener at localhost/34049] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testCompactionWithHFileLink' column='family1' 2023-03-08 19:55:36,861 DEBUG [Listener at localhost/34049] zookeeper.ReadOnlyZKClient(139): Connect 0x3a01f440 to 127.0.0.1:56751 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 19:55:36,865 DEBUG [Listener at localhost/34049] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@17666ce8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 19:55:36,865 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 19:55:36,865 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 19:55:36,865 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 19:55:37,441 DEBUG [RS:0;jenkins-hbase4:33367-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/.tmp/family2/787981f56b5546cc96a6dc32996c9303 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/787981f56b5546cc96a6dc32996c9303 2023-03-08 19:55:37,446 DEBUG [RS:0;jenkins-hbase4:33367-longCompactions-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-03-08 19:55:37,446 INFO [RS:0;jenkins-hbase4:33367-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in d392b5a609f9661af565e4c73dab47f8/family2 of d392b5a609f9661af565e4c73dab47f8 into 787981f56b5546cc96a6dc32996c9303(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 19:55:37,447 DEBUG [RS:0;jenkins-hbase4:33367-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for d392b5a609f9661af565e4c73dab47f8: 2023-03-08 19:55:37,447 INFO [RS:0;jenkins-hbase4:33367-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8., storeName=d392b5a609f9661af565e4c73dab47f8/family2, priority=12, startTime=1678305336580; duration=0sec 2023-03-08 19:55:37,447 DEBUG [RS:0;jenkins-hbase4:33367-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-03-08 19:55:37,448 DEBUG [RS:0;jenkins-hbase4:33367-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-08 19:55:37,450 DEBUG [RS:0;jenkins-hbase4:33367-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-08 19:55:37,450 DEBUG [RS:0;jenkins-hbase4:33367-longCompactions-0] regionserver.HStore(1912): bf25ef5682cfde9aa90a5d6fe8229bdd/family1 is initiating minor compaction (all files) 2023-03-08 19:55:37,450 INFO [RS:0;jenkins-hbase4:33367-longCompactions-0] regionserver.HRegion(2259): Starting compaction of bf25ef5682cfde9aa90a5d6fe8229bdd/family1 in testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. 2023-03-08 19:55:37,450 INFO [RS:0;jenkins-hbase4:33367-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-8cb2d3dd62cc42009c2420fabc400fb6, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-af1975c5b195440797e41fe2eba32cc4, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-046ec2868b184c3e84b6105bf189d3bd, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-c9ebe20caf5d4c2cb57adb8d0820d554] into tmpdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/.tmp, totalSize=21.6 K 2023-03-08 19:55:37,451 DEBUG [RS:0;jenkins-hbase4:33367-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-8cb2d3dd62cc42009c2420fabc400fb6, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678305313716 2023-03-08 19:55:37,451 DEBUG [RS:0;jenkins-hbase4:33367-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-af1975c5b195440797e41fe2eba32cc4, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678305315116 2023-03-08 19:55:37,452 DEBUG [RS:0;jenkins-hbase4:33367-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-046ec2868b184c3e84b6105bf189d3bd, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678305316519 2023-03-08 19:55:37,452 DEBUG [RS:0;jenkins-hbase4:33367-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-c9ebe20caf5d4c2cb57adb8d0820d554, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678305317923 2023-03-08 19:55:37,463 INFO [RS:0;jenkins-hbase4:33367-longCompactions-0] throttle.PressureAwareThroughputController(145): bf25ef5682cfde9aa90a5d6fe8229bdd#family1#compaction#116 average throughput is 1.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 19:55:37,497 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-07773024718d41078416d0bd2e154765, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-3e46bffcb555430ca182ded732797e54, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-6eeb48f439a34bc9ae7493396a8ad303, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-a84f2cc20b89462cab9556cca240ba20] to archive 2023-03-08 19:55:37,498 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:55:37,500 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-07773024718d41078416d0bd2e154765 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-07773024718d41078416d0bd2e154765 2023-03-08 19:55:37,501 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-3e46bffcb555430ca182ded732797e54 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-3e46bffcb555430ca182ded732797e54 2023-03-08 19:55:37,502 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-6eeb48f439a34bc9ae7493396a8ad303 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-6eeb48f439a34bc9ae7493396a8ad303 2023-03-08 19:55:37,503 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-a84f2cc20b89462cab9556cca240ba20 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family2/testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-a84f2cc20b89462cab9556cca240ba20 2023-03-08 19:55:37,740 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308d1d9eba2112a41f2ba509f75df424bed to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230308d1d9eba2112a41f2ba509f75df424bed 2023-03-08 19:55:37,741 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308926fcd8d2fb34a15aa59ceac6ac69281 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230308926fcd8d2fb34a15aa59ceac6ac69281 2023-03-08 19:55:37,741 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303087a3fd8f2f41a4ef8a4baf601872f3246 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303087a3fd8f2f41a4ef8a4baf601872f3246 2023-03-08 19:55:37,741 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 19:55:37,741 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 19:55:37,741 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 19:55:37,756 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 19:55:37,761 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60546, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 19:55:37,773 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 19:55:37,777 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34966, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 19:55:37,792 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230308/family1/79281be0962d47b9adb0cb5d6b132a07 first=Optional[2\x00\x00\x00\x00] last=Optional[2\x00\x00\x00\x07] 2023-03-08 19:55:37,799 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8., hostname=jenkins-hbase4.apache.org,33367,1678305218680, seqNum=32 for row 2 with hfile group [{family1,hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230308/family1/79281be0962d47b9adb0cb5d6b132a07}] 2023-03-08 19:55:37,812 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230308/family1/0e19ed2bffca45dbba451d0bc5ba5041 first=Optional[1\x00\x00\x00\x00] last=Optional[1\x00\x00\x00\x07] 2023-03-08 19:55:37,812 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230308/family1/6bc183ce915f49e7bec68c3e2a2d7a29 first=Optional[0\x00\x00\x00\x00] last=Optional[0\x00\x00\x00\x07] 2023-03-08 19:55:37,813 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33367] regionserver.HStore(763): Validating hfile at hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230308/family1/79281be0962d47b9adb0cb5d6b132a07 for inclusion in d392b5a609f9661af565e4c73dab47f8/family1 2023-03-08 19:55:37,813 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb., hostname=jenkins-hbase4.apache.org,33367,1678305218680, seqNum=32 for row 1 with hfile group [{family1,hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230308/family1/0e19ed2bffca45dbba451d0bc5ba5041}] 2023-03-08 19:55:37,813 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd., hostname=jenkins-hbase4.apache.org,33367,1678305218680, seqNum=32 for row with hfile group [{family1,hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230308/family1/6bc183ce915f49e7bec68c3e2a2d7a29}] 2023-03-08 19:55:37,816 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(763): Validating hfile at hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230308/family1/6bc183ce915f49e7bec68c3e2a2d7a29 for inclusion in bf25ef5682cfde9aa90a5d6fe8229bdd/family1 2023-03-08 19:55:37,816 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33367] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x00 last=2\x00\x00\x00\x07 2023-03-08 19:55:37,816 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33367] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-08 19:55:37,817 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HStore(763): Validating hfile at hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230308/family1/0e19ed2bffca45dbba451d0bc5ba5041 for inclusion in 9c99479945efcf910b16288e607c83eb/family1 2023-03-08 19:55:37,817 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33367] regionserver.HRegion(2446): Flush status journal for d392b5a609f9661af565e4c73dab47f8: 2023-03-08 19:55:37,817 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230308/family1/79281be0962d47b9adb0cb5d6b132a07 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testCompactionWithHFileLink__skgulvqo2boar7rl9sciad5mcqalbucrdun597urelr85md5kcie47qbm7trumlb/family1/79281be0962d47b9adb0cb5d6b132a07 2023-03-08 19:55:37,819 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x00 last=0\x00\x00\x00\x07 2023-03-08 19:55:37,819 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(777): Region bounds: first= last=1 2023-03-08 19:55:37,819 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x00 last=1\x00\x00\x00\x07 2023-03-08 19:55:37,819 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(2446): Flush status journal for bf25ef5682cfde9aa90a5d6fe8229bdd: 2023-03-08 19:55:37,819 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-08 19:55:37,820 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HRegion(2446): Flush status journal for 9c99479945efcf910b16288e607c83eb: 2023-03-08 19:55:37,821 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230308/family1/6bc183ce915f49e7bec68c3e2a2d7a29 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testCompactionWithHFileLink__lsoghcqojpbj1hl0hbuomuhngvuiqfr2li1503fcj5kt4sk8pc04khiuhlvabq8r/family1/6bc183ce915f49e7bec68c3e2a2d7a29 2023-03-08 19:55:37,821 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230308/family1/0e19ed2bffca45dbba451d0bc5ba5041 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testCompactionWithHFileLink__k7rb3pdsp44iqvllvlbhvppde39ajflun3j84di3es3rl43q1psl7it0vu86l41/family1/0e19ed2bffca45dbba451d0bc5ba5041 2023-03-08 19:55:37,822 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33367] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testCompactionWithHFileLink__skgulvqo2boar7rl9sciad5mcqalbucrdun597urelr85md5kcie47qbm7trumlb/family1/79281be0962d47b9adb0cb5d6b132a07 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/317cadd9aa104d94aa7b12bc7ed4aaa1_SeqId_35_ 2023-03-08 19:55:37,823 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33367] regionserver.HStore(849): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testCompactionWithHFileLink__skgulvqo2boar7rl9sciad5mcqalbucrdun597urelr85md5kcie47qbm7trumlb/family1/79281be0962d47b9adb0cb5d6b132a07 into d392b5a609f9661af565e4c73dab47f8/family1 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/317cadd9aa104d94aa7b12bc7ed4aaa1_SeqId_35_ - updating store file list. 2023-03-08 19:55:37,826 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33367] regionserver.HStore(882): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/317cadd9aa104d94aa7b12bc7ed4aaa1_SeqId_35_ into d392b5a609f9661af565e4c73dab47f8/family1 2023-03-08 19:55:37,827 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33367] regionserver.HStore(855): Successfully loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testCompactionWithHFileLink__skgulvqo2boar7rl9sciad5mcqalbucrdun597urelr85md5kcie47qbm7trumlb/family1/79281be0962d47b9adb0cb5d6b132a07 into d392b5a609f9661af565e4c73dab47f8/family1 (new location: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/317cadd9aa104d94aa7b12bc7ed4aaa1_SeqId_35_) 2023-03-08 19:55:37,828 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testCompactionWithHFileLink__skgulvqo2boar7rl9sciad5mcqalbucrdun597urelr85md5kcie47qbm7trumlb/family1/79281be0962d47b9adb0cb5d6b132a07 2023-03-08 19:55:37,830 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testCompactionWithHFileLink__lsoghcqojpbj1hl0hbuomuhngvuiqfr2li1503fcj5kt4sk8pc04khiuhlvabq8r/family1/6bc183ce915f49e7bec68c3e2a2d7a29 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/b7f5a67501344251b4f5dba580ad6967_SeqId_34_ 2023-03-08 19:55:37,830 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(849): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testCompactionWithHFileLink__lsoghcqojpbj1hl0hbuomuhngvuiqfr2li1503fcj5kt4sk8pc04khiuhlvabq8r/family1/6bc183ce915f49e7bec68c3e2a2d7a29 into bf25ef5682cfde9aa90a5d6fe8229bdd/family1 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/b7f5a67501344251b4f5dba580ad6967_SeqId_34_ - updating store file list. 2023-03-08 19:55:37,830 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 19:55:37,831 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testCompactionWithHFileLink__k7rb3pdsp44iqvllvlbhvppde39ajflun3j84di3es3rl43q1psl7it0vu86l41/family1/0e19ed2bffca45dbba451d0bc5ba5041 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/863f8276a7734ab3a9b34dc36dfd8f46_SeqId_34_ 2023-03-08 19:55:37,831 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:55:37,831 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HStore(849): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testCompactionWithHFileLink__k7rb3pdsp44iqvllvlbhvppde39ajflun3j84di3es3rl43q1psl7it0vu86l41/family1/0e19ed2bffca45dbba451d0bc5ba5041 into 9c99479945efcf910b16288e607c83eb/family1 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/863f8276a7734ab3a9b34dc36dfd8f46_SeqId_34_ - updating store file list. 2023-03-08 19:55:37,833 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023030803a51c78c3a9414481e4489a8fc504ad to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023030803a51c78c3a9414481e4489a8fc504ad 2023-03-08 19:55:37,834 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(882): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/b7f5a67501344251b4f5dba580ad6967_SeqId_34_ into bf25ef5682cfde9aa90a5d6fe8229bdd/family1 2023-03-08 19:55:37,834 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(855): Successfully loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testCompactionWithHFileLink__lsoghcqojpbj1hl0hbuomuhngvuiqfr2li1503fcj5kt4sk8pc04khiuhlvabq8r/family1/6bc183ce915f49e7bec68c3e2a2d7a29 into bf25ef5682cfde9aa90a5d6fe8229bdd/family1 (new location: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/b7f5a67501344251b4f5dba580ad6967_SeqId_34_) 2023-03-08 19:55:37,834 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202303088ffe3f5d37194c0fb226cac523d21fef to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202303088ffe3f5d37194c0fb226cac523d21fef 2023-03-08 19:55:37,835 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HStore(882): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/863f8276a7734ab3a9b34dc36dfd8f46_SeqId_34_ into 9c99479945efcf910b16288e607c83eb/family1 2023-03-08 19:55:37,835 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testCompactionWithHFileLink__lsoghcqojpbj1hl0hbuomuhngvuiqfr2li1503fcj5kt4sk8pc04khiuhlvabq8r/family1/6bc183ce915f49e7bec68c3e2a2d7a29 2023-03-08 19:55:37,835 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HStore(855): Successfully loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testCompactionWithHFileLink__k7rb3pdsp44iqvllvlbhvppde39ajflun3j84di3es3rl43q1psl7it0vu86l41/family1/0e19ed2bffca45dbba451d0bc5ba5041 into 9c99479945efcf910b16288e607c83eb/family1 (new location: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/863f8276a7734ab3a9b34dc36dfd8f46_SeqId_34_) 2023-03-08 19:55:37,835 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testCompactionWithHFileLink__k7rb3pdsp44iqvllvlbhvppde39ajflun3j84di3es3rl43q1psl7it0vu86l41/family1/0e19ed2bffca45dbba451d0bc5ba5041 2023-03-08 19:55:37,835 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 19:55:37,836 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230308c5433010db254bcfb902142789f105cc to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230308c5433010db254bcfb902142789f105cc 2023-03-08 19:55:37,837 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230308dd1c4643f5654221b89fb3ab8d9a8eb4 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230308dd1c4643f5654221b89fb3ab8d9a8eb4 2023-03-08 19:55:37,837 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-08 19:55:37,837 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230308' 2023-03-08 19:55:37,837 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 19:55:37,838 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 19:55:37,838 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:55:37,839 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/.tmp/family1/5fddd0ba8f934876afcd76606e82f3d7 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/5fddd0ba8f934876afcd76606e82f3d7 2023-03-08 19:55:37,839 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 5 files of size 29606 starting at candidate #0 after considering 6 permutations with 6 in ratio 2023-03-08 19:55:37,839 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HStore(1912): 9c99479945efcf910b16288e607c83eb/family1 is initiating minor compaction (all files) 2023-03-08 19:55:37,839 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@306386b; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-03-08 19:55:37,840 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202303088d68f70559bb4783aaa77c40b3a3e2c0 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202303088d68f70559bb4783aaa77c40b3a3e2c0 2023-03-08 19:55:37,841 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202303089f52416d158d4f62ac1989a0c5c86166 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202303089f52416d158d4f62ac1989a0c5c86166 2023-03-08 19:55:37,842 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230308 table='testCompactionWithHFileLink' column='family1' 2023-03-08 19:55:37,843 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230308b87a29e85e7140f29524a1dd29216255 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230308b87a29e85e7140f29524a1dd29216255 2023-03-08 19:55:37,843 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:55:37,844 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230308c0451f6624404cb9a261ce5d697fda74 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230308c0451f6624404cb9a261ce5d697fda74 2023-03-08 19:55:37,844 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-08 19:55:37,844 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230308' 2023-03-08 19:55:37,846 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 19:55:37,846 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in d392b5a609f9661af565e4c73dab47f8/family1 of d392b5a609f9661af565e4c73dab47f8 into 5fddd0ba8f934876afcd76606e82f3d7(size=7.6 K), total size for store is 14.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 19:55:37,846 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for d392b5a609f9661af565e4c73dab47f8: 2023-03-08 19:55:37,846 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8., storeName=d392b5a609f9661af565e4c73dab47f8/family1, priority=12, startTime=1678305336579; duration=1sec 2023-03-08 19:55:37,846 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-03-08 19:55:37,846 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 9c99479945efcf910b16288e607c83eb/family1 in testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. 2023-03-08 19:55:37,846 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-123a27e9c7024e02b0fbca054fd86280, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-a433c1e1bdb446f385d1156aa8209307, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-9a28a7cd1d794cde8d25ec94745b3bb9, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-f7d2cb4000d943219c0ae3c05e060127, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/863f8276a7734ab3a9b34dc36dfd8f46_SeqId_34_] into tmpdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/.tmp, totalSize=28.9 K 2023-03-08 19:55:37,846 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202303083653e76c42154508bdba69de9d712f83 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202303083653e76c42154508bdba69de9d712f83 2023-03-08 19:55:37,847 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-123a27e9c7024e02b0fbca054fd86280, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678305313717 2023-03-08 19:55:37,847 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202303088ee4da827de14300899601e052c73946 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202303088ee4da827de14300899601e052c73946 2023-03-08 19:55:37,847 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-a433c1e1bdb446f385d1156aa8209307, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678305315116 2023-03-08 19:55:37,848 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-9a28a7cd1d794cde8d25ec94745b3bb9, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678305316519 2023-03-08 19:55:37,848 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230308d3c2ea31978640cf9022c0e5fb535d1c to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230308d3c2ea31978640cf9022c0e5fb535d1c 2023-03-08 19:55:37,849 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-f7d2cb4000d943219c0ae3c05e060127, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678305317923 2023-03-08 19:55:37,849 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 863f8276a7734ab3a9b34dc36dfd8f46_SeqId_34_, keycount=16, bloomtype=ROW, size=7.3 K, encoding=NONE, compression=NONE, seqNum=35, earliestPutTs=1678305313717 2023-03-08 19:55:37,849 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230308e7a0a9d515b24f63a4013e970fb178a0 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230308e7a0a9d515b24f63a4013e970fb178a0 2023-03-08 19:55:37,849 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-08 19:55:37,849 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230308' 2023-03-08 19:55:37,850 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230308 for table='testCompactionWithHFileLink' and column='family1' 2023-03-08 19:55:37,850 INFO [Listener at localhost/34049] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-03-08 19:55:37,854 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 19:55:37,856 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 19:55:37,863 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 19:55:37,864 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 19:55:37,866 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] throttle.PressureAwareThroughputController(145): 9c99479945efcf910b16288e607c83eb#family1#compaction#117 average throughput is 1.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 19:55:37,871 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 19:55:37,872 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 19:55:37,887 INFO [Listener at localhost/34049] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=285 (was 244) - Thread LEAK? -, OpenFileDescriptor=759 (was 620) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=152 (was 158), ProcessCount=170 (was 170), AvailableMemoryMB=8216 (was 8326) 2023-03-08 19:55:37,898 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-7372b15acde5432d9949bd6d62365bd8, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-e39643d9b2424962b7f3883e1e919f9a, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-c285d649295f485d986c468697edc0ce, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-67ebbe2e123e43d18dd53054062690ec] to archive 2023-03-08 19:55:37,899 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:55:37,899 INFO [Listener at localhost/34049] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompaction Thread=285, OpenFileDescriptor=759, MaxFileDescriptor=60000, SystemLoadAverage=152, ProcessCount=170, AvailableMemoryMB=8216 2023-03-08 19:55:37,901 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-7372b15acde5432d9949bd6d62365bd8 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-7372b15acde5432d9949bd6d62365bd8 2023-03-08 19:55:37,902 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-e39643d9b2424962b7f3883e1e919f9a to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-e39643d9b2424962b7f3883e1e919f9a 2023-03-08 19:55:37,903 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-c285d649295f485d986c468697edc0ce to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-c285d649295f485d986c468697edc0ce 2023-03-08 19:55:37,904 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-67ebbe2e123e43d18dd53054062690ec to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/family1/testCompactionWithHFileLink=d392b5a609f9661af565e4c73dab47f8-67ebbe2e123e43d18dd53054062690ec 2023-03-08 19:55:37,904 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.HMaster$15(3014): Client=jenkins//172.31.14.131 creating {NAME => 'ns'} 2023-03-08 19:55:37,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure2.ProcedureExecutor(1029): Stored pid=104, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=ns 2023-03-08 19:55:37,910 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-03-08 19:55:37,912 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-03-08 19:55:37,915 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, state=SUCCESS; CreateNamespaceProcedure, namespace=ns in 9 msec 2023-03-08 19:55:38,011 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-03-08 19:55:38,013 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-08 19:55:38,014 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=ns:testMinorCompaction 2023-03-08 19:55:38,016 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 19:55:38,016 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "ns" qualifier: "testMinorCompaction" procId is: 105 2023-03-08 19:55:38,016 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 19:55:38,017 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-08 19:55:38,019 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4 2023-03-08 19:55:38,019 DEBUG [HFileArchiver-17] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140 2023-03-08 19:55:38,019 DEBUG [HFileArchiver-18] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6 2023-03-08 19:55:38,019 DEBUG [HFileArchiver-17] backup.HFileArchiver(153): Directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140 empty. 2023-03-08 19:55:38,019 DEBUG [HFileArchiver-18] backup.HFileArchiver(153): Directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6 empty. 2023-03-08 19:55:38,019 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4 empty. 2023-03-08 19:55:38,019 DEBUG [HFileArchiver-17] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140 2023-03-08 19:55:38,020 DEBUG [HFileArchiver-18] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6 2023-03-08 19:55:38,020 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4 2023-03-08 19:55:38,020 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived ns:testMinorCompaction regions 2023-03-08 19:55:38,118 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-08 19:55:38,319 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-08 19:55:38,427 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/ns/testMinorCompaction/.tabledesc/.tableinfo.0000000001 2023-03-08 19:55:38,428 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(7675): creating {ENCODED => 4f3474af7bec67a38b1529f33c13c7a6, NAME => 'ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp 2023-03-08 19:55:38,429 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(7675): creating {ENCODED => 246f7af23020ff6b2e393b5a81b2e3c4, NAME => 'ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp 2023-03-08 19:55:38,429 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(7675): creating {ENCODED => 400484c718b5dfcfabf2e62d3d3e1140, NAME => 'ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp 2023-03-08 19:55:38,620 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-08 19:55:38,680 DEBUG [RS:0;jenkins-hbase4:33367-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/.tmp/family1/bef5d5fc46c94eb582b5cfd023fe2e75 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/bef5d5fc46c94eb582b5cfd023fe2e75 2023-03-08 19:55:38,684 INFO [RS:0;jenkins-hbase4:33367-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in bf25ef5682cfde9aa90a5d6fe8229bdd/family1 of bf25ef5682cfde9aa90a5d6fe8229bdd into bef5d5fc46c94eb582b5cfd023fe2e75(size=7.6 K), total size for store is 14.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 19:55:38,684 DEBUG [RS:0;jenkins-hbase4:33367-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for bf25ef5682cfde9aa90a5d6fe8229bdd: 2023-03-08 19:55:38,684 INFO [RS:0;jenkins-hbase4:33367-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd., storeName=bf25ef5682cfde9aa90a5d6fe8229bdd/family1, priority=12, startTime=1678305336667; duration=1sec 2023-03-08 19:55:38,684 DEBUG [RS:0;jenkins-hbase4:33367-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-03-08 19:55:38,684 DEBUG [RS:0;jenkins-hbase4:33367-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-08 19:55:38,686 DEBUG [RS:0;jenkins-hbase4:33367-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-08 19:55:38,686 DEBUG [RS:0;jenkins-hbase4:33367-longCompactions-0] regionserver.HStore(1912): bf25ef5682cfde9aa90a5d6fe8229bdd/family2 is initiating minor compaction (all files) 2023-03-08 19:55:38,686 INFO [RS:0;jenkins-hbase4:33367-longCompactions-0] regionserver.HRegion(2259): Starting compaction of bf25ef5682cfde9aa90a5d6fe8229bdd/family2 in testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. 2023-03-08 19:55:38,686 INFO [RS:0;jenkins-hbase4:33367-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-044f1e6d183045beae5b75645bb97f60, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-d9b3dfdcafcf4aa3b87dfdc6c038bf9e, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-d97fbd02a2e04d4cb97a3bfe7772fb6e, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-e323979e94844e49a28882288ab3a208] into tmpdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/.tmp, totalSize=20.4 K 2023-03-08 19:55:38,687 DEBUG [RS:0;jenkins-hbase4:33367-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-044f1e6d183045beae5b75645bb97f60, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678305313716 2023-03-08 19:55:38,687 DEBUG [RS:0;jenkins-hbase4:33367-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-d9b3dfdcafcf4aa3b87dfdc6c038bf9e, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678305315116 2023-03-08 19:55:38,688 DEBUG [RS:0;jenkins-hbase4:33367-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-d97fbd02a2e04d4cb97a3bfe7772fb6e, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678305316519 2023-03-08 19:55:38,688 DEBUG [RS:0;jenkins-hbase4:33367-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-e323979e94844e49a28882288ab3a208, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678305317923 2023-03-08 19:55:38,698 INFO [RS:0;jenkins-hbase4:33367-longCompactions-0] throttle.PressureAwareThroughputController(145): bf25ef5682cfde9aa90a5d6fe8229bdd#family2#compaction#118 average throughput is 1.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 19:55:38,703 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-8cb2d3dd62cc42009c2420fabc400fb6, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-af1975c5b195440797e41fe2eba32cc4, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-046ec2868b184c3e84b6105bf189d3bd, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-c9ebe20caf5d4c2cb57adb8d0820d554] to archive 2023-03-08 19:55:38,703 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:55:38,704 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-8cb2d3dd62cc42009c2420fabc400fb6 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-8cb2d3dd62cc42009c2420fabc400fb6 2023-03-08 19:55:38,705 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-af1975c5b195440797e41fe2eba32cc4 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-af1975c5b195440797e41fe2eba32cc4 2023-03-08 19:55:38,706 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-046ec2868b184c3e84b6105bf189d3bd to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-046ec2868b184c3e84b6105bf189d3bd 2023-03-08 19:55:38,707 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-c9ebe20caf5d4c2cb57adb8d0820d554 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family1/testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-c9ebe20caf5d4c2cb57adb8d0820d554 2023-03-08 19:55:38,836 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:55:38,836 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1604): Closing 400484c718b5dfcfabf2e62d3d3e1140, disabling compactions & flushes 2023-03-08 19:55:38,836 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140. 2023-03-08 19:55:38,836 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140. 2023-03-08 19:55:38,836 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140. after waiting 0 ms 2023-03-08 19:55:38,836 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140. 2023-03-08 19:55:38,836 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1838): Closed ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140. 2023-03-08 19:55:38,836 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1558): Region close journal for 400484c718b5dfcfabf2e62d3d3e1140: 2023-03-08 19:55:38,837 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:55:38,837 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1604): Closing 4f3474af7bec67a38b1529f33c13c7a6, disabling compactions & flushes 2023-03-08 19:55:38,838 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6. 2023-03-08 19:55:38,838 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6. 2023-03-08 19:55:38,838 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6. after waiting 0 ms 2023-03-08 19:55:38,838 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6. 2023-03-08 19:55:38,838 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1838): Closed ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6. 2023-03-08 19:55:38,838 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1558): Region close journal for 4f3474af7bec67a38b1529f33c13c7a6: 2023-03-08 19:55:38,841 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:55:38,842 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1604): Closing 246f7af23020ff6b2e393b5a81b2e3c4, disabling compactions & flushes 2023-03-08 19:55:38,842 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4. 2023-03-08 19:55:38,842 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4. 2023-03-08 19:55:38,842 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4. after waiting 0 ms 2023-03-08 19:55:38,842 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4. 2023-03-08 19:55:38,842 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4. 2023-03-08 19:55:38,842 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1558): Region close journal for 246f7af23020ff6b2e393b5a81b2e3c4: 2023-03-08 19:55:38,844 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 19:55:38,844 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1678305338844"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678305338844"}]},"ts":"1678305338844"} 2023-03-08 19:55:38,844 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678305338844"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678305338844"}]},"ts":"1678305338844"} 2023-03-08 19:55:38,844 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678305338844"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678305338844"}]},"ts":"1678305338844"} 2023-03-08 19:55:38,846 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-08 19:55:38,847 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 19:55:38,847 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678305338847"}]},"ts":"1678305338847"} 2023-03-08 19:55:38,848 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLING in hbase:meta 2023-03-08 19:55:38,851 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=246f7af23020ff6b2e393b5a81b2e3c4, ASSIGN}, {pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=400484c718b5dfcfabf2e62d3d3e1140, ASSIGN}, {pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=4f3474af7bec67a38b1529f33c13c7a6, ASSIGN}] 2023-03-08 19:55:38,852 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=4f3474af7bec67a38b1529f33c13c7a6, ASSIGN 2023-03-08 19:55:38,852 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=400484c718b5dfcfabf2e62d3d3e1140, ASSIGN 2023-03-08 19:55:38,853 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=246f7af23020ff6b2e393b5a81b2e3c4, ASSIGN 2023-03-08 19:55:38,853 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=4f3474af7bec67a38b1529f33c13c7a6, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33367,1678305218680; forceNewPlan=false, retain=false 2023-03-08 19:55:38,853 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=400484c718b5dfcfabf2e62d3d3e1140, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33367,1678305218680; forceNewPlan=false, retain=false 2023-03-08 19:55:38,853 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=246f7af23020ff6b2e393b5a81b2e3c4, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33367,1678305218680; forceNewPlan=false, retain=false 2023-03-08 19:55:39,005 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=246f7af23020ff6b2e393b5a81b2e3c4, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:39,005 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=4f3474af7bec67a38b1529f33c13c7a6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:39,005 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=400484c718b5dfcfabf2e62d3d3e1140, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:39,006 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678305339005"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678305339005"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678305339005"}]},"ts":"1678305339005"} 2023-03-08 19:55:39,006 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678305339005"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678305339005"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678305339005"}]},"ts":"1678305339005"} 2023-03-08 19:55:39,006 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1678305339005"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678305339005"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678305339005"}]},"ts":"1678305339005"} 2023-03-08 19:55:39,007 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=109, ppid=108, state=RUNNABLE; OpenRegionProcedure 4f3474af7bec67a38b1529f33c13c7a6, server=jenkins-hbase4.apache.org,33367,1678305218680}] 2023-03-08 19:55:39,008 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=106, state=RUNNABLE; OpenRegionProcedure 246f7af23020ff6b2e393b5a81b2e3c4, server=jenkins-hbase4.apache.org,33367,1678305218680}] 2023-03-08 19:55:39,008 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=107, state=RUNNABLE; OpenRegionProcedure 400484c718b5dfcfabf2e62d3d3e1140, server=jenkins-hbase4.apache.org,33367,1678305218680}] 2023-03-08 19:55:39,084 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/.tmp/family1/e8e2e7c7cac44aceae1f264a97dcff62 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/e8e2e7c7cac44aceae1f264a97dcff62 2023-03-08 19:55:39,088 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 5 (all) file(s) in 9c99479945efcf910b16288e607c83eb/family1 of 9c99479945efcf910b16288e607c83eb into e8e2e7c7cac44aceae1f264a97dcff62(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 19:55:39,088 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 9c99479945efcf910b16288e607c83eb: 2023-03-08 19:55:39,088 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb., storeName=family1, fileCount=5, fileSize=28.9 K (5.4 K, 5.4 K, 5.4 K, 5.4 K, 7.3 K), priority=2, time=1678305337839; duration=1sec 2023-03-08 19:55:39,088 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-03-08 19:55:39,089 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-08 19:55:39,091 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-08 19:55:39,091 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1912): 9c99479945efcf910b16288e607c83eb/family2 is initiating minor compaction (all files) 2023-03-08 19:55:39,091 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 9c99479945efcf910b16288e607c83eb/family2 in testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. 2023-03-08 19:55:39,091 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-cdaeaeb4819a41cc88f69b9970a33dcb, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-68353db2020f4adeba9aeb341538de89, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-7957d21df4a84e90b22385e13d00adde, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-3f39c3b5bff246788da3275e205043fb] into tmpdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/.tmp, totalSize=20.4 K 2023-03-08 19:55:39,092 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-cdaeaeb4819a41cc88f69b9970a33dcb, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678305313717 2023-03-08 19:55:39,092 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-68353db2020f4adeba9aeb341538de89, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678305315116 2023-03-08 19:55:39,093 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-7957d21df4a84e90b22385e13d00adde, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678305316519 2023-03-08 19:55:39,093 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-3f39c3b5bff246788da3275e205043fb, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678305317923 2023-03-08 19:55:39,096 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 19:55:39,102 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-123a27e9c7024e02b0fbca054fd86280, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-a433c1e1bdb446f385d1156aa8209307, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-9a28a7cd1d794cde8d25ec94745b3bb9, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-f7d2cb4000d943219c0ae3c05e060127, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/863f8276a7734ab3a9b34dc36dfd8f46_SeqId_34_] to archive 2023-03-08 19:55:39,103 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:55:39,105 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-123a27e9c7024e02b0fbca054fd86280 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-123a27e9c7024e02b0fbca054fd86280 2023-03-08 19:55:39,107 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-a433c1e1bdb446f385d1156aa8209307 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-a433c1e1bdb446f385d1156aa8209307 2023-03-08 19:55:39,108 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-9a28a7cd1d794cde8d25ec94745b3bb9 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-9a28a7cd1d794cde8d25ec94745b3bb9 2023-03-08 19:55:39,110 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-f7d2cb4000d943219c0ae3c05e060127 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-f7d2cb4000d943219c0ae3c05e060127 2023-03-08 19:55:39,111 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/863f8276a7734ab3a9b34dc36dfd8f46_SeqId_34_ to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family1/863f8276a7734ab3a9b34dc36dfd8f46_SeqId_34_ 2023-03-08 19:55:39,120 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] throttle.PressureAwareThroughputController(145): 9c99479945efcf910b16288e607c83eb#family2#compaction#119 average throughput is 0.15 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 19:55:39,121 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-08 19:55:39,161 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6. 2023-03-08 19:55:39,161 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 4f3474af7bec67a38b1529f33c13c7a6, NAME => 'ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6.', STARTKEY => '2', ENDKEY => ''} 2023-03-08 19:55:39,162 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 4f3474af7bec67a38b1529f33c13c7a6 2023-03-08 19:55:39,162 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:55:39,162 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 4f3474af7bec67a38b1529f33c13c7a6 2023-03-08 19:55:39,162 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 4f3474af7bec67a38b1529f33c13c7a6 2023-03-08 19:55:39,163 INFO [StoreOpener-4f3474af7bec67a38b1529f33c13c7a6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 4f3474af7bec67a38b1529f33c13c7a6 2023-03-08 19:55:39,164 DEBUG [StoreOpener-4f3474af7bec67a38b1529f33c13c7a6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family1 2023-03-08 19:55:39,164 DEBUG [StoreOpener-4f3474af7bec67a38b1529f33c13c7a6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family1 2023-03-08 19:55:39,164 INFO [StoreOpener-4f3474af7bec67a38b1529f33c13c7a6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4f3474af7bec67a38b1529f33c13c7a6 columnFamilyName family1 2023-03-08 19:55:39,165 INFO [StoreOpener-4f3474af7bec67a38b1529f33c13c7a6-1] regionserver.HStore(310): Store=4f3474af7bec67a38b1529f33c13c7a6/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:55:39,165 INFO [StoreOpener-4f3474af7bec67a38b1529f33c13c7a6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 4f3474af7bec67a38b1529f33c13c7a6 2023-03-08 19:55:39,166 DEBUG [StoreOpener-4f3474af7bec67a38b1529f33c13c7a6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family2 2023-03-08 19:55:39,166 DEBUG [StoreOpener-4f3474af7bec67a38b1529f33c13c7a6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family2 2023-03-08 19:55:39,167 INFO [StoreOpener-4f3474af7bec67a38b1529f33c13c7a6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4f3474af7bec67a38b1529f33c13c7a6 columnFamilyName family2 2023-03-08 19:55:39,168 INFO [StoreOpener-4f3474af7bec67a38b1529f33c13c7a6-1] regionserver.HStore(310): Store=4f3474af7bec67a38b1529f33c13c7a6/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:55:39,168 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6 2023-03-08 19:55:39,169 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6 2023-03-08 19:55:39,170 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 19:55:39,171 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 4f3474af7bec67a38b1529f33c13c7a6 2023-03-08 19:55:39,173 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 19:55:39,173 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 4f3474af7bec67a38b1529f33c13c7a6; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72201985, jitterRate=0.07589341700077057}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 19:55:39,173 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 4f3474af7bec67a38b1529f33c13c7a6: 2023-03-08 19:55:39,174 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6., pid=109, masterSystemTime=1678305339159 2023-03-08 19:55:39,175 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6. 2023-03-08 19:55:39,175 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6. 2023-03-08 19:55:39,175 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4. 2023-03-08 19:55:39,175 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 246f7af23020ff6b2e393b5a81b2e3c4, NAME => 'ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4.', STARTKEY => '', ENDKEY => '1'} 2023-03-08 19:55:39,175 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=4f3474af7bec67a38b1529f33c13c7a6, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:39,176 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 246f7af23020ff6b2e393b5a81b2e3c4 2023-03-08 19:55:39,176 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678305339175"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678305339175"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678305339175"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678305339175"}]},"ts":"1678305339175"} 2023-03-08 19:55:39,176 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:55:39,176 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 246f7af23020ff6b2e393b5a81b2e3c4 2023-03-08 19:55:39,176 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 246f7af23020ff6b2e393b5a81b2e3c4 2023-03-08 19:55:39,177 INFO [StoreOpener-246f7af23020ff6b2e393b5a81b2e3c4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 246f7af23020ff6b2e393b5a81b2e3c4 2023-03-08 19:55:39,178 DEBUG [StoreOpener-246f7af23020ff6b2e393b5a81b2e3c4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family1 2023-03-08 19:55:39,178 DEBUG [StoreOpener-246f7af23020ff6b2e393b5a81b2e3c4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family1 2023-03-08 19:55:39,178 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=109, resume processing ppid=108 2023-03-08 19:55:39,178 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=109, ppid=108, state=SUCCESS; OpenRegionProcedure 4f3474af7bec67a38b1529f33c13c7a6, server=jenkins-hbase4.apache.org,33367,1678305218680 in 170 msec 2023-03-08 19:55:39,178 INFO [StoreOpener-246f7af23020ff6b2e393b5a81b2e3c4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 246f7af23020ff6b2e393b5a81b2e3c4 columnFamilyName family1 2023-03-08 19:55:39,179 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=108, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=4f3474af7bec67a38b1529f33c13c7a6, ASSIGN in 327 msec 2023-03-08 19:55:39,179 INFO [StoreOpener-246f7af23020ff6b2e393b5a81b2e3c4-1] regionserver.HStore(310): Store=246f7af23020ff6b2e393b5a81b2e3c4/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:55:39,179 INFO [StoreOpener-246f7af23020ff6b2e393b5a81b2e3c4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 246f7af23020ff6b2e393b5a81b2e3c4 2023-03-08 19:55:39,180 DEBUG [StoreOpener-246f7af23020ff6b2e393b5a81b2e3c4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family2 2023-03-08 19:55:39,180 DEBUG [StoreOpener-246f7af23020ff6b2e393b5a81b2e3c4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family2 2023-03-08 19:55:39,181 INFO [StoreOpener-246f7af23020ff6b2e393b5a81b2e3c4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 246f7af23020ff6b2e393b5a81b2e3c4 columnFamilyName family2 2023-03-08 19:55:39,181 INFO [StoreOpener-246f7af23020ff6b2e393b5a81b2e3c4-1] regionserver.HStore(310): Store=246f7af23020ff6b2e393b5a81b2e3c4/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:55:39,182 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4 2023-03-08 19:55:39,182 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4 2023-03-08 19:55:39,183 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 19:55:39,184 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 246f7af23020ff6b2e393b5a81b2e3c4 2023-03-08 19:55:39,185 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 19:55:39,186 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 246f7af23020ff6b2e393b5a81b2e3c4; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59400819, jitterRate=-0.11485882103443146}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 19:55:39,186 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 246f7af23020ff6b2e393b5a81b2e3c4: 2023-03-08 19:55:39,186 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4., pid=110, masterSystemTime=1678305339159 2023-03-08 19:55:39,187 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4. 2023-03-08 19:55:39,187 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4. 2023-03-08 19:55:39,187 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140. 2023-03-08 19:55:39,188 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 400484c718b5dfcfabf2e62d3d3e1140, NAME => 'ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140.', STARTKEY => '1', ENDKEY => '2'} 2023-03-08 19:55:39,188 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=246f7af23020ff6b2e393b5a81b2e3c4, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:39,188 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 400484c718b5dfcfabf2e62d3d3e1140 2023-03-08 19:55:39,188 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:55:39,188 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678305339188"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678305339188"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678305339188"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678305339188"}]},"ts":"1678305339188"} 2023-03-08 19:55:39,188 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 400484c718b5dfcfabf2e62d3d3e1140 2023-03-08 19:55:39,188 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 400484c718b5dfcfabf2e62d3d3e1140 2023-03-08 19:55:39,189 INFO [StoreOpener-400484c718b5dfcfabf2e62d3d3e1140-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 400484c718b5dfcfabf2e62d3d3e1140 2023-03-08 19:55:39,190 DEBUG [StoreOpener-400484c718b5dfcfabf2e62d3d3e1140-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family1 2023-03-08 19:55:39,190 DEBUG [StoreOpener-400484c718b5dfcfabf2e62d3d3e1140-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family1 2023-03-08 19:55:39,190 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=106 2023-03-08 19:55:39,190 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=106, state=SUCCESS; OpenRegionProcedure 246f7af23020ff6b2e393b5a81b2e3c4, server=jenkins-hbase4.apache.org,33367,1678305218680 in 181 msec 2023-03-08 19:55:39,191 INFO [StoreOpener-400484c718b5dfcfabf2e62d3d3e1140-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 400484c718b5dfcfabf2e62d3d3e1140 columnFamilyName family1 2023-03-08 19:55:39,191 INFO [StoreOpener-400484c718b5dfcfabf2e62d3d3e1140-1] regionserver.HStore(310): Store=400484c718b5dfcfabf2e62d3d3e1140/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:55:39,191 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=246f7af23020ff6b2e393b5a81b2e3c4, ASSIGN in 339 msec 2023-03-08 19:55:39,192 INFO [StoreOpener-400484c718b5dfcfabf2e62d3d3e1140-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 400484c718b5dfcfabf2e62d3d3e1140 2023-03-08 19:55:39,193 DEBUG [StoreOpener-400484c718b5dfcfabf2e62d3d3e1140-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family2 2023-03-08 19:55:39,193 DEBUG [StoreOpener-400484c718b5dfcfabf2e62d3d3e1140-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family2 2023-03-08 19:55:39,193 INFO [StoreOpener-400484c718b5dfcfabf2e62d3d3e1140-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 400484c718b5dfcfabf2e62d3d3e1140 columnFamilyName family2 2023-03-08 19:55:39,193 INFO [StoreOpener-400484c718b5dfcfabf2e62d3d3e1140-1] regionserver.HStore(310): Store=400484c718b5dfcfabf2e62d3d3e1140/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:55:39,194 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140 2023-03-08 19:55:39,194 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140 2023-03-08 19:55:39,196 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 19:55:39,196 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 400484c718b5dfcfabf2e62d3d3e1140 2023-03-08 19:55:39,198 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 19:55:39,198 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 400484c718b5dfcfabf2e62d3d3e1140; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72260958, jitterRate=0.07677218317985535}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 19:55:39,198 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 400484c718b5dfcfabf2e62d3d3e1140: 2023-03-08 19:55:39,200 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140., pid=111, masterSystemTime=1678305339159 2023-03-08 19:55:39,201 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140. 2023-03-08 19:55:39,201 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140. 2023-03-08 19:55:39,201 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=400484c718b5dfcfabf2e62d3d3e1140, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:39,202 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1678305339201"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678305339201"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678305339201"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678305339201"}]},"ts":"1678305339201"} 2023-03-08 19:55:39,204 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=107 2023-03-08 19:55:39,204 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=107, state=SUCCESS; OpenRegionProcedure 400484c718b5dfcfabf2e62d3d3e1140, server=jenkins-hbase4.apache.org,33367,1678305218680 in 195 msec 2023-03-08 19:55:39,205 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=105 2023-03-08 19:55:39,205 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=400484c718b5dfcfabf2e62d3d3e1140, ASSIGN in 353 msec 2023-03-08 19:55:39,206 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 19:55:39,206 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678305339206"}]},"ts":"1678305339206"} 2023-03-08 19:55:39,207 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLED in hbase:meta 2023-03-08 19:55:39,209 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 19:55:39,210 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; CreateTableProcedure table=ns:testMinorCompaction in 1.1960 sec 2023-03-08 19:55:39,929 DEBUG [RS:0;jenkins-hbase4:33367-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/.tmp/family2/b25ee91ef79340d690c8c00fde5d9bc1 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/b25ee91ef79340d690c8c00fde5d9bc1 2023-03-08 19:55:39,933 INFO [RS:0;jenkins-hbase4:33367-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in bf25ef5682cfde9aa90a5d6fe8229bdd/family2 of bf25ef5682cfde9aa90a5d6fe8229bdd into b25ee91ef79340d690c8c00fde5d9bc1(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 19:55:39,933 DEBUG [RS:0;jenkins-hbase4:33367-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for bf25ef5682cfde9aa90a5d6fe8229bdd: 2023-03-08 19:55:39,933 INFO [RS:0;jenkins-hbase4:33367-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd., storeName=bf25ef5682cfde9aa90a5d6fe8229bdd/family2, priority=12, startTime=1678305336667; duration=1sec 2023-03-08 19:55:39,933 DEBUG [RS:0;jenkins-hbase4:33367-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-03-08 19:55:39,933 DEBUG [RS:0;jenkins-hbase4:33367-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-03-08 19:55:39,933 DEBUG [RS:0;jenkins-hbase4:33367-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-03-08 19:55:39,934 DEBUG [RS:0;jenkins-hbase4:33367-longCompactions-0] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-03-08 19:55:39,934 DEBUG [RS:0;jenkins-hbase4:33367-longCompactions-0] regionserver.CompactSplit(415): Not compacting testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. because compaction request was cancelled 2023-03-08 19:55:39,998 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-044f1e6d183045beae5b75645bb97f60, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-d9b3dfdcafcf4aa3b87dfdc6c038bf9e, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-d97fbd02a2e04d4cb97a3bfe7772fb6e, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-e323979e94844e49a28882288ab3a208] to archive 2023-03-08 19:55:39,999 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:55:40,000 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-044f1e6d183045beae5b75645bb97f60 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-044f1e6d183045beae5b75645bb97f60 2023-03-08 19:55:40,002 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-d9b3dfdcafcf4aa3b87dfdc6c038bf9e to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-d9b3dfdcafcf4aa3b87dfdc6c038bf9e 2023-03-08 19:55:40,002 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-d97fbd02a2e04d4cb97a3bfe7772fb6e to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-d97fbd02a2e04d4cb97a3bfe7772fb6e 2023-03-08 19:55:40,003 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-e323979e94844e49a28882288ab3a208 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/family2/testCompactionWithHFileLink=bf25ef5682cfde9aa90a5d6fe8229bdd-e323979e94844e49a28882288ab3a208 2023-03-08 19:55:40,123 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-08 19:55:40,123 INFO [Listener at localhost/34049] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: ns:testMinorCompaction, procId: 105 completed 2023-03-08 19:55:40,129 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 19:55:40,129 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 19:55:40,129 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 19:55:40,131 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:55:40,133 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-08 19:55:40,133 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-08 19:55:40,133 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:55:40,133 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-08 19:55:40,133 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-08 19:55:40,133 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:40,133 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 19:55:40,135 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:40,135 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:55:40,135 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:55:40,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:55:40,136 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:40,136 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:55:40,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 19:55:40,136 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:40,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:55:40,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 19:55:40,137 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-08 19:55:40,137 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-08 19:55:40,137 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-08 19:55:40,137 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:55:40,137 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-08 19:55:40,137 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4. 2023-03-08 19:55:40,137 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 19:55:40,137 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:55:40,137 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140. 2023-03-08 19:55:40,137 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6. 2023-03-08 19:55:40,137 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4. started... 2023-03-08 19:55:40,137 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6. started... 2023-03-08 19:55:40,137 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140. started... 2023-03-08 19:55:40,139 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 246f7af23020ff6b2e393b5a81b2e3c4 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-08 19:55:40,139 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 4f3474af7bec67a38b1529f33c13c7a6 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-08 19:55:40,139 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 400484c718b5dfcfabf2e62d3d3e1140 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-08 19:55:40,153 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303084a3646332df14691ac2039b01851dcad to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303084a3646332df14691ac2039b01851dcad 2023-03-08 19:55:40,153 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303088305dee3f3864f32b14bffe13cf965fe to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202303088305dee3f3864f32b14bffe13cf965fe 2023-03-08 19:55:40,339 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/.tmp/family2/4fe7eccf5d4d41d59098ae42504c066b as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/4fe7eccf5d4d41d59098ae42504c066b 2023-03-08 19:55:40,342 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 9c99479945efcf910b16288e607c83eb/family2 of 9c99479945efcf910b16288e607c83eb into 4fe7eccf5d4d41d59098ae42504c066b(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 19:55:40,342 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 9c99479945efcf910b16288e607c83eb: 2023-03-08 19:55:40,342 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb., storeName=9c99479945efcf910b16288e607c83eb/family2, priority=12, startTime=1678305336750; duration=1sec 2023-03-08 19:55:40,343 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 19:55:40,404 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-cdaeaeb4819a41cc88f69b9970a33dcb, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-68353db2020f4adeba9aeb341538de89, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-7957d21df4a84e90b22385e13d00adde, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-3f39c3b5bff246788da3275e205043fb] to archive 2023-03-08 19:55:40,405 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:55:40,407 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-cdaeaeb4819a41cc88f69b9970a33dcb to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-cdaeaeb4819a41cc88f69b9970a33dcb 2023-03-08 19:55:40,408 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-68353db2020f4adeba9aeb341538de89 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-68353db2020f4adeba9aeb341538de89 2023-03-08 19:55:40,409 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-7957d21df4a84e90b22385e13d00adde to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-7957d21df4a84e90b22385e13d00adde 2023-03-08 19:55:40,410 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-3f39c3b5bff246788da3275e205043fb to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/family2/testCompactionWithHFileLink=9c99479945efcf910b16288e607c83eb-3f39c3b5bff246788da3275e205043fb 2023-03-08 19:55:40,552 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308a10ad2f7870249df9a0e3e77cff902d3 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230308a10ad2f7870249df9a0e3e77cff902d3 2023-03-08 19:55:40,559 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/.tmp/family1/0aff56a404254297a28cc5fbe8e9d96c 2023-03-08 19:55:40,559 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/.tmp/family1/df669bc209bd4727824a56f1911e07d5 2023-03-08 19:55:40,805 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-03-08 19:55:40,957 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/.tmp/family1/5fa158d2f5ac42adbb65d4c5f21e174f 2023-03-08 19:55:40,978 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308846de27c59c1461097d01836a9832c2d to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230308846de27c59c1461097d01836a9832c2d 2023-03-08 19:55:40,979 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303081fe06a4a0fe444be9432c8f1ad6b6c1d to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c202303081fe06a4a0fe444be9432c8f1ad6b6c1d 2023-03-08 19:55:41,373 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023030875869b0caa664422a124bdf1bb42f9dc to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e2023030875869b0caa664422a124bdf1bb42f9dc 2023-03-08 19:55:41,383 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/.tmp/family2/c6024a067fce4e1d92bbac35b4e28e0b 2023-03-08 19:55:41,383 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/.tmp/family2/6eebe7cea5834c5c8077435b56e8286e 2023-03-08 19:55:41,387 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/.tmp/family1/0aff56a404254297a28cc5fbe8e9d96c as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family1/0aff56a404254297a28cc5fbe8e9d96c 2023-03-08 19:55:41,387 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/.tmp/family1/df669bc209bd4727824a56f1911e07d5 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family1/df669bc209bd4727824a56f1911e07d5 2023-03-08 19:55:41,391 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family1/df669bc209bd4727824a56f1911e07d5, entries=4, sequenceid=5, filesize=5.4 K 2023-03-08 19:55:41,391 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family1/0aff56a404254297a28cc5fbe8e9d96c, entries=4, sequenceid=5, filesize=5.4 K 2023-03-08 19:55:41,392 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/.tmp/family2/6eebe7cea5834c5c8077435b56e8286e as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family2/6eebe7cea5834c5c8077435b56e8286e 2023-03-08 19:55:41,392 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/.tmp/family2/c6024a067fce4e1d92bbac35b4e28e0b as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family2/c6024a067fce4e1d92bbac35b4e28e0b 2023-03-08 19:55:41,395 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family2/6eebe7cea5834c5c8077435b56e8286e, entries=2, sequenceid=5, filesize=5.1 K 2023-03-08 19:55:41,395 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family2/c6024a067fce4e1d92bbac35b4e28e0b, entries=2, sequenceid=5, filesize=5.1 K 2023-03-08 19:55:41,396 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 4f3474af7bec67a38b1529f33c13c7a6 in 1257ms, sequenceid=5, compaction requested=false 2023-03-08 19:55:41,396 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 400484c718b5dfcfabf2e62d3d3e1140 in 1257ms, sequenceid=5, compaction requested=false 2023-03-08 19:55:41,396 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'ns:testMinorCompaction' 2023-03-08 19:55:41,396 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'ns:testMinorCompaction' 2023-03-08 19:55:41,398 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 400484c718b5dfcfabf2e62d3d3e1140: 2023-03-08 19:55:41,398 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 4f3474af7bec67a38b1529f33c13c7a6: 2023-03-08 19:55:41,398 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140. 2023-03-08 19:55:41,398 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6. 2023-03-08 19:55:41,398 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 19:55:41,398 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 19:55:41,778 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/.tmp/family2/0f1ad37649af4c38986688792d5ea9cd 2023-03-08 19:55:41,782 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/.tmp/family1/5fa158d2f5ac42adbb65d4c5f21e174f as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family1/5fa158d2f5ac42adbb65d4c5f21e174f 2023-03-08 19:55:41,785 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family1/5fa158d2f5ac42adbb65d4c5f21e174f, entries=4, sequenceid=5, filesize=5.4 K 2023-03-08 19:55:41,786 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/.tmp/family2/0f1ad37649af4c38986688792d5ea9cd as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family2/0f1ad37649af4c38986688792d5ea9cd 2023-03-08 19:55:41,789 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family2/0f1ad37649af4c38986688792d5ea9cd, entries=2, sequenceid=5, filesize=5.1 K 2023-03-08 19:55:41,789 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 246f7af23020ff6b2e393b5a81b2e3c4 in 1650ms, sequenceid=5, compaction requested=false 2023-03-08 19:55:41,790 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 246f7af23020ff6b2e393b5a81b2e3c4: 2023-03-08 19:55:41,790 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4. 2023-03-08 19:55:41,790 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 19:55:41,790 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 19:55:41,790 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:41,790 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-08 19:55:41,790 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-08 19:55:41,792 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:41,792 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 19:55:41,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:41,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:41,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:41,792 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 19:55:41,792 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:55:41,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:41,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:41,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 19:55:41,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:41,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:41,793 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-08 19:55:41,793 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@29bf2707[Count = 0] remaining members to acquire global barrier 2023-03-08 19:55:41,793 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-08 19:55:41,794 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 19:55:41,795 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 19:55:41,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 19:55:41,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 19:55:41,795 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:41,795 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:55:41,795 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-08 19:55:41,795 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-08 19:55:41,795 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:55:41,796 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:41,796 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-08 19:55:41,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:41,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:41,796 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:55:41,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:41,797 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-08 19:55:41,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:41,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:41,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 19:55:41,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:41,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:41,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 19:55:41,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:41,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:55:41,799 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-08 19:55:41,799 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-08 19:55:41,799 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:55:41,799 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:55:41,799 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-08 19:55:41,799 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:55:41,801 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:41,801 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:41,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:41,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:41,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:41,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:41,801 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:55:41,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:41,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:41,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:55:41,802 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:41,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:55:41,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 19:55:41,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:41,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:41,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 19:55:41,803 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:41,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:41,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:41,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 19:55:41,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:41,805 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:41,805 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:55:41,805 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 19:55:41,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:55:41,805 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-03-08 19:55:41,805 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 19:55:41,805 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-08 19:55:41,805 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:55:41,805 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:55:41,805 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:55:41,805 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:41,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:55:41,806 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 19:55:41,806 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 19:55:41,806 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-08 19:55:41,806 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:41,806 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:55:41,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:55:41,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:55:41,906 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:55:41,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:55:41,908 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 19:55:41,908 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 19:55:41,908 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 19:55:41,911 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:55:41,912 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-08 19:55:41,913 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-08 19:55:41,913 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-08 19:55:41,913 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:55:41,913 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-08 19:55:41,913 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-08 19:55:41,913 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:41,913 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 19:55:41,915 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:41,915 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:55:41,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:55:41,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:55:41,915 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:41,915 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:55:41,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 19:55:41,915 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:41,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:55:41,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 19:55:41,916 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-08 19:55:41,916 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-08 19:55:41,916 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-08 19:55:41,916 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-08 19:55:41,916 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:55:41,916 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-08 19:55:41,916 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4. 2023-03-08 19:55:41,916 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 19:55:41,916 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6. 2023-03-08 19:55:41,916 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140. 2023-03-08 19:55:41,916 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140. started... 2023-03-08 19:55:41,916 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4. started... 2023-03-08 19:55:41,916 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6. started... 2023-03-08 19:55:41,916 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 400484c718b5dfcfabf2e62d3d3e1140 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-08 19:55:41,917 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 246f7af23020ff6b2e393b5a81b2e3c4 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-08 19:55:41,917 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 4f3474af7bec67a38b1529f33c13c7a6 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-08 19:55:41,916 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:55:41,934 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308e5fe5179111449e78799cf38b3b8dd02 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230308e5fe5179111449e78799cf38b3b8dd02 2023-03-08 19:55:41,937 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308d9d77e23b0a94979a8ca909a7c82f855 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230308d9d77e23b0a94979a8ca909a7c82f855 2023-03-08 19:55:41,939 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/.tmp/family1/737ae079e14c4549b973fdbab6b5dc11 2023-03-08 19:55:42,334 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303080295d1df141342e29b3ff89245807d21 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303080295d1df141342e29b3ff89245807d21 2023-03-08 19:55:42,341 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/.tmp/family1/c8660a197d4e465a845342ad5882ab0b 2023-03-08 19:55:42,352 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308a57418b3cd7b415cb1829e53368b8532 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230308a57418b3cd7b415cb1829e53368b8532 2023-03-08 19:55:42,740 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/.tmp/family1/abeed6b454f944428982b4f3cd3547ef 2023-03-08 19:55:42,753 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308d64d45c0bd454799976b0b22fcef1e0a to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230308d64d45c0bd454799976b0b22fcef1e0a 2023-03-08 19:55:42,755 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303083281ae9155c547d4a52fd0ddaa37b4af to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202303083281ae9155c547d4a52fd0ddaa37b4af 2023-03-08 19:55:42,768 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/.tmp/family2/fe4c4767549b45dc876c04af7a545a63 2023-03-08 19:55:42,772 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/.tmp/family1/737ae079e14c4549b973fdbab6b5dc11 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family1/737ae079e14c4549b973fdbab6b5dc11 2023-03-08 19:55:42,776 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family1/737ae079e14c4549b973fdbab6b5dc11, entries=4, sequenceid=9, filesize=5.4 K 2023-03-08 19:55:42,776 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/.tmp/family2/fe4c4767549b45dc876c04af7a545a63 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family2/fe4c4767549b45dc876c04af7a545a63 2023-03-08 19:55:42,779 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family2/fe4c4767549b45dc876c04af7a545a63, entries=2, sequenceid=9, filesize=5.1 K 2023-03-08 19:55:42,780 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 4f3474af7bec67a38b1529f33c13c7a6 in 863ms, sequenceid=9, compaction requested=false 2023-03-08 19:55:42,780 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 4f3474af7bec67a38b1529f33c13c7a6: 2023-03-08 19:55:42,780 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6. 2023-03-08 19:55:42,780 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 19:55:43,161 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/.tmp/family2/770ad5c4ffa44d6db4aecb2ee5d6b968 2023-03-08 19:55:43,166 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/.tmp/family1/c8660a197d4e465a845342ad5882ab0b as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family1/c8660a197d4e465a845342ad5882ab0b 2023-03-08 19:55:43,170 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family1/c8660a197d4e465a845342ad5882ab0b, entries=4, sequenceid=9, filesize=5.4 K 2023-03-08 19:55:43,170 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/.tmp/family2/f071f8fcf7d5486590dcfdbc1b759bbc 2023-03-08 19:55:43,170 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/.tmp/family2/770ad5c4ffa44d6db4aecb2ee5d6b968 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family2/770ad5c4ffa44d6db4aecb2ee5d6b968 2023-03-08 19:55:43,174 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family2/770ad5c4ffa44d6db4aecb2ee5d6b968, entries=2, sequenceid=9, filesize=5.1 K 2023-03-08 19:55:43,174 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/.tmp/family1/abeed6b454f944428982b4f3cd3547ef as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family1/abeed6b454f944428982b4f3cd3547ef 2023-03-08 19:55:43,175 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 246f7af23020ff6b2e393b5a81b2e3c4 in 1259ms, sequenceid=9, compaction requested=false 2023-03-08 19:55:43,175 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 246f7af23020ff6b2e393b5a81b2e3c4: 2023-03-08 19:55:43,175 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4. 2023-03-08 19:55:43,175 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 19:55:43,178 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family1/abeed6b454f944428982b4f3cd3547ef, entries=4, sequenceid=9, filesize=5.4 K 2023-03-08 19:55:43,179 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/.tmp/family2/f071f8fcf7d5486590dcfdbc1b759bbc as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family2/f071f8fcf7d5486590dcfdbc1b759bbc 2023-03-08 19:55:43,182 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family2/f071f8fcf7d5486590dcfdbc1b759bbc, entries=2, sequenceid=9, filesize=5.1 K 2023-03-08 19:55:43,183 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 400484c718b5dfcfabf2e62d3d3e1140 in 1267ms, sequenceid=9, compaction requested=false 2023-03-08 19:55:43,183 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 400484c718b5dfcfabf2e62d3d3e1140: 2023-03-08 19:55:43,183 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140. 2023-03-08 19:55:43,183 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 19:55:43,183 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 19:55:43,184 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:43,184 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-08 19:55:43,184 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-08 19:55:43,185 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:43,185 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 19:55:43,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:43,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:43,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:43,186 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 19:55:43,186 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:55:43,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:43,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:43,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 19:55:43,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:43,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:43,187 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-08 19:55:43,187 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@67812f53[Count = 0] remaining members to acquire global barrier 2023-03-08 19:55:43,187 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-08 19:55:43,187 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 19:55:43,189 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 19:55:43,189 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 19:55:43,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 19:55:43,189 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-08 19:55:43,189 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-08 19:55:43,189 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:55:43,190 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:43,190 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:55:43,193 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:43,193 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-08 19:55:43,194 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:55:43,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:43,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:43,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:43,194 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-08 19:55:43,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:43,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:43,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 19:55:43,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:43,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:43,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 19:55:43,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:43,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:55:43,196 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-08 19:55:43,196 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-08 19:55:43,196 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:55:43,196 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:55:43,196 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-08 19:55:43,196 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:55:43,197 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:43,197 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:43,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:43,198 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:55:43,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:43,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:43,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:43,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:43,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:55:43,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:55:43,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:43,198 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:43,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:43,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 19:55:43,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:43,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 19:55:43,199 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:43,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:43,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:43,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 19:55:43,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:43,202 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:43,202 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 19:55:43,202 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 19:55:43,202 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:55:43,202 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:43,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:55:43,202 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-03-08 19:55:43,202 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:55:43,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:55:43,202 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-08 19:55:43,203 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:55:43,202 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 19:55:43,203 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 19:55:43,202 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:55:43,203 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:43,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:55:43,203 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-08 19:55:43,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:55:43,203 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:55:43,303 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:55:43,304 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:55:43,305 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 19:55:43,305 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 19:55:43,306 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 19:55:43,308 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:55:43,309 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-08 19:55:43,309 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-08 19:55:43,309 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-08 19:55:43,309 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:55:43,309 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-08 19:55:43,310 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-08 19:55:43,310 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:43,310 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 19:55:43,311 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:55:43,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:55:43,311 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:43,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:55:43,312 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:43,312 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:55:43,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 19:55:43,312 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:43,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:55:43,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 19:55:43,312 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-08 19:55:43,313 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-08 19:55:43,313 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-08 19:55:43,313 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-08 19:55:43,313 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:55:43,313 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-08 19:55:43,313 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4. 2023-03-08 19:55:43,313 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4. started... 2023-03-08 19:55:43,313 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140. 2023-03-08 19:55:43,313 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6. 2023-03-08 19:55:43,313 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140. started... 2023-03-08 19:55:43,313 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 19:55:43,313 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:55:43,313 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 246f7af23020ff6b2e393b5a81b2e3c4 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-08 19:55:43,313 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6. started... 2023-03-08 19:55:43,313 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 400484c718b5dfcfabf2e62d3d3e1140 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-08 19:55:43,314 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 4f3474af7bec67a38b1529f33c13c7a6 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-08 19:55:43,733 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303087a3d4e11c3114daa94a2f779e93815a7 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202303087a3d4e11c3114daa94a2f779e93815a7 2023-03-08 19:55:43,733 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023030838fc9e69b2d54538bca78b71c82f81f5 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023030838fc9e69b2d54538bca78b71c82f81f5 2023-03-08 19:55:43,733 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308c5e19a40398e4816822fa76bd9362f4d to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230308c5e19a40398e4816822fa76bd9362f4d 2023-03-08 19:55:44,141 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/.tmp/family1/27939c96e77146dba9e259a2ad228d4f 2023-03-08 19:55:44,141 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/.tmp/family1/38146755cbe748c187b8021a02c70dde 2023-03-08 19:55:44,141 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/.tmp/family1/05273ced8cba402c8ea6f09863d347dd 2023-03-08 19:55:44,154 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308517a54f4f19747698063de1baee3a6a8 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230308517a54f4f19747698063de1baee3a6a8 2023-03-08 19:55:44,555 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308c8702627983249fb8d97a0307191605d to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230308c8702627983249fb8d97a0307191605d 2023-03-08 19:55:44,555 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308ab033f204c924d83a7288e45701042ba to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230308ab033f204c924d83a7288e45701042ba 2023-03-08 19:55:44,558 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/.tmp/family2/b29fa78b7d5948f89aa61ce292c34b8d 2023-03-08 19:55:44,560 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/.tmp/family2/dd0980f17a12400e8602a4106a652670 2023-03-08 19:55:44,563 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/.tmp/family1/38146755cbe748c187b8021a02c70dde as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family1/38146755cbe748c187b8021a02c70dde 2023-03-08 19:55:44,563 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/.tmp/family1/05273ced8cba402c8ea6f09863d347dd as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family1/05273ced8cba402c8ea6f09863d347dd 2023-03-08 19:55:44,567 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family1/38146755cbe748c187b8021a02c70dde, entries=4, sequenceid=13, filesize=5.4 K 2023-03-08 19:55:44,567 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family1/05273ced8cba402c8ea6f09863d347dd, entries=4, sequenceid=13, filesize=5.4 K 2023-03-08 19:55:44,567 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/.tmp/family2/b29fa78b7d5948f89aa61ce292c34b8d as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family2/b29fa78b7d5948f89aa61ce292c34b8d 2023-03-08 19:55:44,568 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/.tmp/family2/dd0980f17a12400e8602a4106a652670 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family2/dd0980f17a12400e8602a4106a652670 2023-03-08 19:55:44,570 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family2/b29fa78b7d5948f89aa61ce292c34b8d, entries=2, sequenceid=13, filesize=5.1 K 2023-03-08 19:55:44,571 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 4f3474af7bec67a38b1529f33c13c7a6 in 1257ms, sequenceid=13, compaction requested=true 2023-03-08 19:55:44,571 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family2/dd0980f17a12400e8602a4106a652670, entries=2, sequenceid=13, filesize=5.1 K 2023-03-08 19:55:44,571 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 4f3474af7bec67a38b1529f33c13c7a6: 2023-03-08 19:55:44,571 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6. 2023-03-08 19:55:44,571 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 19:55:44,572 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 246f7af23020ff6b2e393b5a81b2e3c4 in 1259ms, sequenceid=13, compaction requested=true 2023-03-08 19:55:44,572 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 246f7af23020ff6b2e393b5a81b2e3c4: 2023-03-08 19:55:44,572 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4. 2023-03-08 19:55:44,572 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 19:55:44,663 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 19:55:44,703 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 19:55:44,703 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3a01f440 to 127.0.0.1:56751 2023-03-08 19:55:44,703 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 19:55:44,703 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 19:55:44,704 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x00d77f15 to 127.0.0.1:56751 2023-03-08 19:55:44,704 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 19:55:44,960 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/.tmp/family2/641ff8bc83ea420ab9efa11895eb599b 2023-03-08 19:55:44,964 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/.tmp/family1/27939c96e77146dba9e259a2ad228d4f as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family1/27939c96e77146dba9e259a2ad228d4f 2023-03-08 19:55:44,967 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family1/27939c96e77146dba9e259a2ad228d4f, entries=4, sequenceid=13, filesize=5.4 K 2023-03-08 19:55:44,968 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/.tmp/family2/641ff8bc83ea420ab9efa11895eb599b as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family2/641ff8bc83ea420ab9efa11895eb599b 2023-03-08 19:55:44,971 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family2/641ff8bc83ea420ab9efa11895eb599b, entries=2, sequenceid=13, filesize=5.1 K 2023-03-08 19:55:44,972 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 400484c718b5dfcfabf2e62d3d3e1140 in 1659ms, sequenceid=13, compaction requested=true 2023-03-08 19:55:44,972 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 400484c718b5dfcfabf2e62d3d3e1140: 2023-03-08 19:55:44,972 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140. 2023-03-08 19:55:44,972 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 19:55:44,972 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 19:55:44,972 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:44,972 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-08 19:55:44,972 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-08 19:55:44,980 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:44,980 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 19:55:44,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:44,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:44,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:44,980 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 19:55:44,980 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:55:44,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:44,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:44,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 19:55:44,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:44,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:44,982 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-08 19:55:44,982 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@e121af2[Count = 0] remaining members to acquire global barrier 2023-03-08 19:55:44,982 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-08 19:55:44,982 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 19:55:44,983 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 19:55:44,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 19:55:44,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 19:55:44,983 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-08 19:55:44,983 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-08 19:55:44,983 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:55:44,984 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-08 19:55:44,984 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:44,984 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:55:44,984 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-08 19:55:44,985 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-08 19:55:44,985 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:55:44,985 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-08 19:55:44,985 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:55:44,985 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:55:44,985 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-08 19:55:44,985 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:55:44,986 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:44,986 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:44,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:44,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:44,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:44,986 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:44,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:44,986 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:55:44,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:44,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:55:44,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:55:44,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 19:55:44,987 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:44,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:44,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:44,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 19:55:44,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:44,989 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:44,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:44,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 19:55:44,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:44,994 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:44,994 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:55:44,994 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 19:55:44,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:55:44,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:55:44,995 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:55:44,994 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:55:44,995 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-08 19:55:44,995 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:55:44,995 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 19:55:44,995 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:44,995 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 19:55:44,995 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 19:55:44,995 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:44,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:55:44,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:55:44,995 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-08 19:55:44,995 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:55:45,095 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:55:45,096 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:55:45,098 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 19:55:45,098 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 19:55:45,098 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 19:55:45,100 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:55:45,102 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-08 19:55:45,102 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-08 19:55:45,102 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-08 19:55:45,102 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:55:45,102 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-08 19:55:45,102 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-08 19:55:45,103 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:45,103 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 19:55:45,104 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:45,104 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:55:45,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:55:45,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:55:45,104 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:45,105 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:55:45,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 19:55:45,105 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:45,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:55:45,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 19:55:45,105 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-08 19:55:45,106 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-08 19:55:45,106 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-08 19:55:45,106 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-08 19:55:45,106 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:55:45,106 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-08 19:55:45,106 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4. 2023-03-08 19:55:45,106 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4. started... 2023-03-08 19:55:45,106 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140. 2023-03-08 19:55:45,106 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140. started... 2023-03-08 19:55:45,106 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 19:55:45,106 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:55:45,106 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 400484c718b5dfcfabf2e62d3d3e1140 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-08 19:55:45,106 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 246f7af23020ff6b2e393b5a81b2e3c4 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-08 19:55:45,106 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6. 2023-03-08 19:55:45,106 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6. started... 2023-03-08 19:55:45,107 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 4f3474af7bec67a38b1529f33c13c7a6 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-08 19:55:45,523 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023030876bef706fb7042daa63ac87037fa3418 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023030876bef706fb7042daa63ac87037fa3418 2023-03-08 19:55:45,523 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303086da63d9e7908497eab040ecec89a84e4 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303086da63d9e7908497eab040ecec89a84e4 2023-03-08 19:55:45,523 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303081b73e5eb4c284443a2ae0630ea65d560 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202303081b73e5eb4c284443a2ae0630ea65d560 2023-03-08 19:55:45,533 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/.tmp/family1/dd30a634535f4f678061a8d8786b9811 2023-03-08 19:55:45,928 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/.tmp/family1/6cd7993c15d141709b53eda02a9d1e53 2023-03-08 19:55:45,928 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/.tmp/family1/aa7a1846a5944a5e965959b94b915c5e 2023-03-08 19:55:45,943 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303081582cc54eacf48679a6270eab0c265c6 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202303081582cc54eacf48679a6270eab0c265c6 2023-03-08 19:55:46,344 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308caf6e318a382449f90c2baf98d203040 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230308caf6e318a382449f90c2baf98d203040 2023-03-08 19:55:46,344 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308d79f170501fd47b7b220df0e1a974216 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230308d79f170501fd47b7b220df0e1a974216 2023-03-08 19:55:46,348 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/.tmp/family2/13198ce3b79f49c2a88f7ce1c2b5351b 2023-03-08 19:55:46,352 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/.tmp/family1/dd30a634535f4f678061a8d8786b9811 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family1/dd30a634535f4f678061a8d8786b9811 2023-03-08 19:55:46,355 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family1/dd30a634535f4f678061a8d8786b9811, entries=4, sequenceid=17, filesize=5.4 K 2023-03-08 19:55:46,356 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/.tmp/family2/13198ce3b79f49c2a88f7ce1c2b5351b as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family2/13198ce3b79f49c2a88f7ce1c2b5351b 2023-03-08 19:55:46,359 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family2/13198ce3b79f49c2a88f7ce1c2b5351b, entries=2, sequenceid=17, filesize=5.1 K 2023-03-08 19:55:46,360 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 246f7af23020ff6b2e393b5a81b2e3c4 in 1254ms, sequenceid=17, compaction requested=true 2023-03-08 19:55:46,360 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 246f7af23020ff6b2e393b5a81b2e3c4: 2023-03-08 19:55:46,360 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4. 2023-03-08 19:55:46,360 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 19:55:46,749 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/.tmp/family2/2757b342b5f745aaa6edd352f5f6e416 2023-03-08 19:55:46,749 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/.tmp/family2/aa2ed92083484cff9293d5453ddbe91b 2023-03-08 19:55:46,753 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/.tmp/family1/aa7a1846a5944a5e965959b94b915c5e as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family1/aa7a1846a5944a5e965959b94b915c5e 2023-03-08 19:55:46,753 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/.tmp/family1/6cd7993c15d141709b53eda02a9d1e53 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family1/6cd7993c15d141709b53eda02a9d1e53 2023-03-08 19:55:46,756 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family1/aa7a1846a5944a5e965959b94b915c5e, entries=4, sequenceid=17, filesize=5.4 K 2023-03-08 19:55:46,757 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family1/6cd7993c15d141709b53eda02a9d1e53, entries=4, sequenceid=17, filesize=5.4 K 2023-03-08 19:55:46,757 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/.tmp/family2/aa2ed92083484cff9293d5453ddbe91b as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family2/aa2ed92083484cff9293d5453ddbe91b 2023-03-08 19:55:46,758 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/.tmp/family2/2757b342b5f745aaa6edd352f5f6e416 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family2/2757b342b5f745aaa6edd352f5f6e416 2023-03-08 19:55:46,761 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family2/2757b342b5f745aaa6edd352f5f6e416, entries=2, sequenceid=17, filesize=5.1 K 2023-03-08 19:55:46,761 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family2/aa2ed92083484cff9293d5453ddbe91b, entries=2, sequenceid=17, filesize=5.1 K 2023-03-08 19:55:46,762 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 400484c718b5dfcfabf2e62d3d3e1140 in 1656ms, sequenceid=17, compaction requested=true 2023-03-08 19:55:46,762 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 4f3474af7bec67a38b1529f33c13c7a6 in 1655ms, sequenceid=17, compaction requested=true 2023-03-08 19:55:46,762 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 400484c718b5dfcfabf2e62d3d3e1140: 2023-03-08 19:55:46,762 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 4f3474af7bec67a38b1529f33c13c7a6: 2023-03-08 19:55:46,762 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140. 2023-03-08 19:55:46,762 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6. 2023-03-08 19:55:46,762 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 19:55:46,762 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 19:55:46,762 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 19:55:46,762 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:46,762 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-08 19:55:46,762 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-08 19:55:46,764 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:46,764 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 19:55:46,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:46,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:46,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:46,764 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 19:55:46,764 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:55:46,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:46,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:46,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 19:55:46,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:46,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:46,765 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-08 19:55:46,765 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@c4acd85[Count = 0] remaining members to acquire global barrier 2023-03-08 19:55:46,765 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-08 19:55:46,766 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 19:55:46,767 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 19:55:46,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 19:55:46,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 19:55:46,767 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-08 19:55:46,767 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:46,767 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:55:46,767 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-08 19:55:46,767 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:55:46,769 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:46,769 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-08 19:55:46,769 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:55:46,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:46,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:46,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:46,769 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-08 19:55:46,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:46,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:46,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 19:55:46,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:46,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:46,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 19:55:46,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:46,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:55:46,771 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-08 19:55:46,771 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-08 19:55:46,771 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:55:46,771 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:55:46,771 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-08 19:55:46,771 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:55:46,773 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:46,773 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:46,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:46,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:46,773 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:55:46,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:46,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:46,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:46,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:55:46,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:55:46,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:46,773 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:46,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:46,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 19:55:46,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:46,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 19:55:46,774 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:46,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:46,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:46,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 19:55:46,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:46,776 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:46,776 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 19:55:46,776 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 19:55:46,776 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:55:46,776 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:46,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:55:46,777 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-08 19:55:46,777 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:55:46,777 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:55:46,777 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 19:55:46,776 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:55:46,777 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-08 19:55:46,777 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 19:55:46,777 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:55:46,777 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-03-08 19:55:46,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:55:46,777 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:46,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:55:46,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:55:46,877 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:55:46,878 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:55:46,889 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 19:55:46,899 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 19:55:46,906 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 19:55:46,907 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 19:55:46,934 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:55:46,936 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-08 19:55:46,936 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-08 19:55:46,936 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-08 19:55:46,936 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:55:46,936 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-08 19:55:46,937 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-08 19:55:46,937 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:46,937 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 19:55:46,938 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:46,938 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:55:46,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:55:46,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:55:46,939 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:46,939 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:55:46,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 19:55:46,939 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:46,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:55:46,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 19:55:46,939 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-08 19:55:46,939 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-08 19:55:46,940 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-08 19:55:46,940 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-08 19:55:46,940 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:55:46,940 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-08 19:55:46,940 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4. 2023-03-08 19:55:46,940 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4. started... 2023-03-08 19:55:46,940 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140. 2023-03-08 19:55:46,940 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6. 2023-03-08 19:55:46,940 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 246f7af23020ff6b2e393b5a81b2e3c4 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-08 19:55:46,940 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 19:55:46,940 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:55:46,940 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6. started... 2023-03-08 19:55:46,940 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140. started... 2023-03-08 19:55:46,940 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 4f3474af7bec67a38b1529f33c13c7a6 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-08 19:55:46,940 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 400484c718b5dfcfabf2e62d3d3e1140 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-08 19:55:47,763 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/.tmp/family1/1c45040ef6344a49b84cd677bb2137f0 2023-03-08 19:55:47,763 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/.tmp/family1/cb0b84fb6a4c49e5b9412efabc8ed725 2023-03-08 19:55:47,763 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/.tmp/family1/68ccebb17c4e4533a14934d301b006ff 2023-03-08 19:55:47,767 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1c45040ef6344a49b84cd677bb2137f0 2023-03-08 19:55:47,767 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 68ccebb17c4e4533a14934d301b006ff 2023-03-08 19:55:47,767 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cb0b84fb6a4c49e5b9412efabc8ed725 2023-03-08 19:55:47,783 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/.tmp/family2/04a341827886490189ae1e8d45ac4e26 2023-03-08 19:55:47,786 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 04a341827886490189ae1e8d45ac4e26 2023-03-08 19:55:47,787 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/.tmp/family1/1c45040ef6344a49b84cd677bb2137f0 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family1/1c45040ef6344a49b84cd677bb2137f0 2023-03-08 19:55:47,790 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1c45040ef6344a49b84cd677bb2137f0 2023-03-08 19:55:47,790 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family1/1c45040ef6344a49b84cd677bb2137f0, entries=3, sequenceid=23, filesize=5.0 K 2023-03-08 19:55:47,791 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/.tmp/family2/04a341827886490189ae1e8d45ac4e26 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family2/04a341827886490189ae1e8d45ac4e26 2023-03-08 19:55:47,794 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 04a341827886490189ae1e8d45ac4e26 2023-03-08 19:55:47,794 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family2/04a341827886490189ae1e8d45ac4e26, entries=1, sequenceid=23, filesize=5.0 K 2023-03-08 19:55:47,795 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 246f7af23020ff6b2e393b5a81b2e3c4 in 855ms, sequenceid=23, compaction requested=true 2023-03-08 19:55:47,795 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 246f7af23020ff6b2e393b5a81b2e3c4: 2023-03-08 19:55:47,795 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4. 2023-03-08 19:55:47,795 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 19:55:48,583 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/.tmp/family2/7c289eda1e664923ade347b489cf449c 2023-03-08 19:55:48,583 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/.tmp/family2/09b77a2d02c74eeb9c815270281a9bb7 2023-03-08 19:55:48,586 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 09b77a2d02c74eeb9c815270281a9bb7 2023-03-08 19:55:48,586 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7c289eda1e664923ade347b489cf449c 2023-03-08 19:55:48,587 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/.tmp/family1/cb0b84fb6a4c49e5b9412efabc8ed725 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family1/cb0b84fb6a4c49e5b9412efabc8ed725 2023-03-08 19:55:48,587 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/.tmp/family1/68ccebb17c4e4533a14934d301b006ff as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family1/68ccebb17c4e4533a14934d301b006ff 2023-03-08 19:55:48,590 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cb0b84fb6a4c49e5b9412efabc8ed725 2023-03-08 19:55:48,590 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family1/cb0b84fb6a4c49e5b9412efabc8ed725, entries=3, sequenceid=23, filesize=5.0 K 2023-03-08 19:55:48,590 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 68ccebb17c4e4533a14934d301b006ff 2023-03-08 19:55:48,590 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family1/68ccebb17c4e4533a14934d301b006ff, entries=3, sequenceid=23, filesize=5.0 K 2023-03-08 19:55:48,591 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/.tmp/family2/09b77a2d02c74eeb9c815270281a9bb7 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family2/09b77a2d02c74eeb9c815270281a9bb7 2023-03-08 19:55:48,591 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/.tmp/family2/7c289eda1e664923ade347b489cf449c as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family2/7c289eda1e664923ade347b489cf449c 2023-03-08 19:55:48,594 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 09b77a2d02c74eeb9c815270281a9bb7 2023-03-08 19:55:48,594 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7c289eda1e664923ade347b489cf449c 2023-03-08 19:55:48,594 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family2/09b77a2d02c74eeb9c815270281a9bb7, entries=1, sequenceid=23, filesize=5.0 K 2023-03-08 19:55:48,594 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family2/7c289eda1e664923ade347b489cf449c, entries=1, sequenceid=23, filesize=5.0 K 2023-03-08 19:55:48,595 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 400484c718b5dfcfabf2e62d3d3e1140 in 1655ms, sequenceid=23, compaction requested=true 2023-03-08 19:55:48,595 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 4f3474af7bec67a38b1529f33c13c7a6 in 1655ms, sequenceid=23, compaction requested=true 2023-03-08 19:55:48,595 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 400484c718b5dfcfabf2e62d3d3e1140: 2023-03-08 19:55:48,595 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 4f3474af7bec67a38b1529f33c13c7a6: 2023-03-08 19:55:48,595 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140. 2023-03-08 19:55:48,595 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6. 2023-03-08 19:55:48,595 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 19:55:48,595 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 19:55:48,595 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 19:55:48,595 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:48,595 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-08 19:55:48,595 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-08 19:55:48,598 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 19:55:48,598 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:48,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:48,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:48,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:48,598 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 19:55:48,598 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:55:48,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:48,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:48,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 19:55:48,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:48,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:48,599 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-08 19:55:48,599 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5c9d5414[Count = 0] remaining members to acquire global barrier 2023-03-08 19:55:48,599 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-08 19:55:48,599 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 19:55:48,600 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 19:55:48,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 19:55:48,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 19:55:48,601 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-08 19:55:48,601 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:48,601 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:55:48,601 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-08 19:55:48,601 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:55:48,605 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:48,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:48,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:48,605 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-08 19:55:48,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:48,605 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:55:48,605 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-08 19:55:48,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:48,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:48,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 19:55:48,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:48,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:48,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 19:55:48,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:48,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:55:48,607 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-08 19:55:48,607 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-08 19:55:48,608 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:55:48,608 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:55:48,608 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-08 19:55:48,608 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:55:48,610 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:48,610 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:48,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:48,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:48,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:48,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:48,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:48,610 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:55:48,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:48,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:55:48,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:55:48,611 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:48,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 19:55:48,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:48,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:48,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 19:55:48,611 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:48,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:48,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:48,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-08 19:55:48,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:48,614 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:48,614 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:55:48,614 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 19:55:48,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:55:48,614 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-08 19:55:48,614 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:55:48,614 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:55:48,614 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-03-08 19:55:48,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:55:48,614 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:55:48,614 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-08 19:55:48,614 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:48,614 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 19:55:48,614 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-08 19:55:48,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:55:48,614 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:55:48,614 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-08 19:55:48,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:55:48,614 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-08 19:55:48,714 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:55:48,715 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:55:48,716 DEBUG [Listener at localhost/34049] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 19:55:48,716 DEBUG [Listener at localhost/34049] regionserver.HStore(1912): 246f7af23020ff6b2e393b5a81b2e3c4/family1 is initiating major compaction (all files) 2023-03-08 19:55:48,716 INFO [Listener at localhost/34049] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 19:55:48,716 INFO [Listener at localhost/34049] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 19:55:48,716 INFO [Listener at localhost/34049] regionserver.HRegion(2259): Starting compaction of 246f7af23020ff6b2e393b5a81b2e3c4/family1 in ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4. 2023-03-08 19:55:48,716 INFO [Listener at localhost/34049] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family1/5fa158d2f5ac42adbb65d4c5f21e174f, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family1/c8660a197d4e465a845342ad5882ab0b, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family1/05273ced8cba402c8ea6f09863d347dd, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family1/dd30a634535f4f678061a8d8786b9811, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family1/1c45040ef6344a49b84cd677bb2137f0] into tmpdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/.tmp, totalSize=26.6 K 2023-03-08 19:55:48,717 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 5fa158d2f5ac42adbb65d4c5f21e174f, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678305340129 2023-03-08 19:55:48,718 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting c8660a197d4e465a845342ad5882ab0b, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678305341908 2023-03-08 19:55:48,718 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 05273ced8cba402c8ea6f09863d347dd, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678305343305 2023-03-08 19:55:48,718 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting dd30a634535f4f678061a8d8786b9811, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678305345098 2023-03-08 19:55:48,719 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 1c45040ef6344a49b84cd677bb2137f0, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 19:55:48,728 INFO [Listener at localhost/34049] throttle.PressureAwareThroughputController(145): 246f7af23020ff6b2e393b5a81b2e3c4#family1#compaction#150 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 19:55:49,135 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230308f0465298ca354f6c91462fc234440c76_del 2023-03-08 19:55:49,136 INFO [Listener at localhost/34049] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308f0465298ca354f6c91462fc234440c76_del to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230308f0465298ca354f6c91462fc234440c76_del 2023-03-08 19:55:49,949 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 77f015f0bd014cd4b0aaa762ac039f94 2023-03-08 19:55:49,949 DEBUG [Listener at localhost/34049] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/.tmp/family1/77f015f0bd014cd4b0aaa762ac039f94 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family1/77f015f0bd014cd4b0aaa762ac039f94 2023-03-08 19:55:49,952 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 77f015f0bd014cd4b0aaa762ac039f94 2023-03-08 19:55:49,953 INFO [Listener at localhost/34049] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 246f7af23020ff6b2e393b5a81b2e3c4/family1 of 246f7af23020ff6b2e393b5a81b2e3c4 into 77f015f0bd014cd4b0aaa762ac039f94(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 19:55:49,953 DEBUG [Listener at localhost/34049] regionserver.HRegion(2289): Compaction status journal for 246f7af23020ff6b2e393b5a81b2e3c4: 2023-03-08 19:55:49,953 DEBUG [Listener at localhost/34049] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 19:55:49,953 DEBUG [Listener at localhost/34049] regionserver.HStore(1912): 246f7af23020ff6b2e393b5a81b2e3c4/family2 is initiating major compaction (all files) 2023-03-08 19:55:49,954 INFO [Listener at localhost/34049] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 19:55:49,954 INFO [Listener at localhost/34049] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 19:55:49,954 INFO [Listener at localhost/34049] regionserver.HRegion(2259): Starting compaction of 246f7af23020ff6b2e393b5a81b2e3c4/family2 in ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4. 2023-03-08 19:55:49,954 INFO [Listener at localhost/34049] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family2/0f1ad37649af4c38986688792d5ea9cd, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family2/770ad5c4ffa44d6db4aecb2ee5d6b968, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family2/dd0980f17a12400e8602a4106a652670, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family2/13198ce3b79f49c2a88f7ce1c2b5351b, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family2/04a341827886490189ae1e8d45ac4e26] into tmpdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/.tmp, totalSize=25.3 K 2023-03-08 19:55:49,955 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 0f1ad37649af4c38986688792d5ea9cd, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678305340129 2023-03-08 19:55:49,955 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 770ad5c4ffa44d6db4aecb2ee5d6b968, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678305341908 2023-03-08 19:55:49,956 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting dd0980f17a12400e8602a4106a652670, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678305343305 2023-03-08 19:55:49,956 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 13198ce3b79f49c2a88f7ce1c2b5351b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678305345098 2023-03-08 19:55:49,956 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 04a341827886490189ae1e8d45ac4e26, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 19:55:49,964 INFO [Listener at localhost/34049] throttle.PressureAwareThroughputController(145): 246f7af23020ff6b2e393b5a81b2e3c4#family2#compaction#151 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 19:55:50,002 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family1/5fa158d2f5ac42adbb65d4c5f21e174f, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family1/c8660a197d4e465a845342ad5882ab0b, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family1/05273ced8cba402c8ea6f09863d347dd, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family1/dd30a634535f4f678061a8d8786b9811, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family1/1c45040ef6344a49b84cd677bb2137f0] to archive 2023-03-08 19:55:50,003 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:55:50,004 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family1/5fa158d2f5ac42adbb65d4c5f21e174f to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family1/5fa158d2f5ac42adbb65d4c5f21e174f 2023-03-08 19:55:50,005 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family1/c8660a197d4e465a845342ad5882ab0b to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family1/c8660a197d4e465a845342ad5882ab0b 2023-03-08 19:55:50,006 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family1/05273ced8cba402c8ea6f09863d347dd to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family1/05273ced8cba402c8ea6f09863d347dd 2023-03-08 19:55:50,007 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family1/dd30a634535f4f678061a8d8786b9811 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family1/dd30a634535f4f678061a8d8786b9811 2023-03-08 19:55:50,008 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family1/1c45040ef6344a49b84cd677bb2137f0 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family1/1c45040ef6344a49b84cd677bb2137f0 2023-03-08 19:55:50,379 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230308ffbe684694af4e07b35bcc3016428dac_del 2023-03-08 19:55:50,379 INFO [Listener at localhost/34049] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308ffbe684694af4e07b35bcc3016428dac_del to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230308ffbe684694af4e07b35bcc3016428dac_del 2023-03-08 19:55:51,192 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0ef5f8c0ba20471f89a2d54b9116153b 2023-03-08 19:55:51,193 DEBUG [Listener at localhost/34049] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/.tmp/family2/0ef5f8c0ba20471f89a2d54b9116153b as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family2/0ef5f8c0ba20471f89a2d54b9116153b 2023-03-08 19:55:51,196 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0ef5f8c0ba20471f89a2d54b9116153b 2023-03-08 19:55:51,197 INFO [Listener at localhost/34049] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 246f7af23020ff6b2e393b5a81b2e3c4/family2 of 246f7af23020ff6b2e393b5a81b2e3c4 into 0ef5f8c0ba20471f89a2d54b9116153b(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 19:55:51,198 DEBUG [Listener at localhost/34049] regionserver.HRegion(2289): Compaction status journal for 246f7af23020ff6b2e393b5a81b2e3c4: 2023-03-08 19:55:51,198 DEBUG [Listener at localhost/34049] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 19:55:51,198 DEBUG [Listener at localhost/34049] regionserver.HStore(1912): 400484c718b5dfcfabf2e62d3d3e1140/family1 is initiating major compaction (all files) 2023-03-08 19:55:51,198 INFO [Listener at localhost/34049] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 19:55:51,198 INFO [Listener at localhost/34049] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 19:55:51,199 INFO [Listener at localhost/34049] regionserver.HRegion(2259): Starting compaction of 400484c718b5dfcfabf2e62d3d3e1140/family1 in ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140. 2023-03-08 19:55:51,199 INFO [Listener at localhost/34049] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family1/0aff56a404254297a28cc5fbe8e9d96c, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family1/abeed6b454f944428982b4f3cd3547ef, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family1/27939c96e77146dba9e259a2ad228d4f, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family1/6cd7993c15d141709b53eda02a9d1e53, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family1/cb0b84fb6a4c49e5b9412efabc8ed725] into tmpdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/.tmp, totalSize=26.6 K 2023-03-08 19:55:51,199 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 0aff56a404254297a28cc5fbe8e9d96c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678305340129 2023-03-08 19:55:51,200 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting abeed6b454f944428982b4f3cd3547ef, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678305341908 2023-03-08 19:55:51,200 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 27939c96e77146dba9e259a2ad228d4f, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678305343305 2023-03-08 19:55:51,201 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 6cd7993c15d141709b53eda02a9d1e53, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678305345098 2023-03-08 19:55:51,201 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting cb0b84fb6a4c49e5b9412efabc8ed725, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 19:55:51,210 INFO [Listener at localhost/34049] throttle.PressureAwareThroughputController(145): 400484c718b5dfcfabf2e62d3d3e1140#family1#compaction#152 average throughput is 0.55 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 19:55:51,224 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202303084620368a7a13468193cde834527f10c5_del 2023-03-08 19:55:51,224 INFO [Listener at localhost/34049] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303084620368a7a13468193cde834527f10c5_del to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303084620368a7a13468193cde834527f10c5_del 2023-03-08 19:55:51,304 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family2/0f1ad37649af4c38986688792d5ea9cd, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family2/770ad5c4ffa44d6db4aecb2ee5d6b968, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family2/dd0980f17a12400e8602a4106a652670, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family2/13198ce3b79f49c2a88f7ce1c2b5351b, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family2/04a341827886490189ae1e8d45ac4e26] to archive 2023-03-08 19:55:51,305 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:55:51,306 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family2/0f1ad37649af4c38986688792d5ea9cd to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family2/0f1ad37649af4c38986688792d5ea9cd 2023-03-08 19:55:51,307 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family2/770ad5c4ffa44d6db4aecb2ee5d6b968 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family2/770ad5c4ffa44d6db4aecb2ee5d6b968 2023-03-08 19:55:51,308 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family2/dd0980f17a12400e8602a4106a652670 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family2/dd0980f17a12400e8602a4106a652670 2023-03-08 19:55:51,309 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family2/13198ce3b79f49c2a88f7ce1c2b5351b to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family2/13198ce3b79f49c2a88f7ce1c2b5351b 2023-03-08 19:55:51,310 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family2/04a341827886490189ae1e8d45ac4e26 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family2/04a341827886490189ae1e8d45ac4e26 2023-03-08 19:55:52,045 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fa788ddcd4814adb908695645e1098d9 2023-03-08 19:55:52,046 DEBUG [Listener at localhost/34049] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/.tmp/family1/fa788ddcd4814adb908695645e1098d9 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family1/fa788ddcd4814adb908695645e1098d9 2023-03-08 19:55:52,049 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fa788ddcd4814adb908695645e1098d9 2023-03-08 19:55:52,050 INFO [Listener at localhost/34049] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 400484c718b5dfcfabf2e62d3d3e1140/family1 of 400484c718b5dfcfabf2e62d3d3e1140 into fa788ddcd4814adb908695645e1098d9(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 19:55:52,050 DEBUG [Listener at localhost/34049] regionserver.HRegion(2289): Compaction status journal for 400484c718b5dfcfabf2e62d3d3e1140: 2023-03-08 19:55:52,050 DEBUG [Listener at localhost/34049] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 19:55:52,050 DEBUG [Listener at localhost/34049] regionserver.HStore(1912): 400484c718b5dfcfabf2e62d3d3e1140/family2 is initiating major compaction (all files) 2023-03-08 19:55:52,050 INFO [Listener at localhost/34049] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 19:55:52,051 INFO [Listener at localhost/34049] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 19:55:52,051 INFO [Listener at localhost/34049] regionserver.HRegion(2259): Starting compaction of 400484c718b5dfcfabf2e62d3d3e1140/family2 in ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140. 2023-03-08 19:55:52,051 INFO [Listener at localhost/34049] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family2/c6024a067fce4e1d92bbac35b4e28e0b, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family2/f071f8fcf7d5486590dcfdbc1b759bbc, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family2/641ff8bc83ea420ab9efa11895eb599b, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family2/2757b342b5f745aaa6edd352f5f6e416, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family2/09b77a2d02c74eeb9c815270281a9bb7] into tmpdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/.tmp, totalSize=25.3 K 2023-03-08 19:55:52,051 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting c6024a067fce4e1d92bbac35b4e28e0b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678305340129 2023-03-08 19:55:52,051 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting f071f8fcf7d5486590dcfdbc1b759bbc, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678305341908 2023-03-08 19:55:52,052 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 641ff8bc83ea420ab9efa11895eb599b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678305343305 2023-03-08 19:55:52,052 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 2757b342b5f745aaa6edd352f5f6e416, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678305345098 2023-03-08 19:55:52,052 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 09b77a2d02c74eeb9c815270281a9bb7, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 19:55:52,060 INFO [Listener at localhost/34049] throttle.PressureAwareThroughputController(145): 400484c718b5dfcfabf2e62d3d3e1140#family2#compaction#153 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 19:55:52,098 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family1/0aff56a404254297a28cc5fbe8e9d96c, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family1/abeed6b454f944428982b4f3cd3547ef, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family1/27939c96e77146dba9e259a2ad228d4f, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family1/6cd7993c15d141709b53eda02a9d1e53, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family1/cb0b84fb6a4c49e5b9412efabc8ed725] to archive 2023-03-08 19:55:52,099 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:55:52,100 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family1/0aff56a404254297a28cc5fbe8e9d96c to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family1/0aff56a404254297a28cc5fbe8e9d96c 2023-03-08 19:55:52,101 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family1/abeed6b454f944428982b4f3cd3547ef to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family1/abeed6b454f944428982b4f3cd3547ef 2023-03-08 19:55:52,102 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family1/27939c96e77146dba9e259a2ad228d4f to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family1/27939c96e77146dba9e259a2ad228d4f 2023-03-08 19:55:52,103 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family1/6cd7993c15d141709b53eda02a9d1e53 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family1/6cd7993c15d141709b53eda02a9d1e53 2023-03-08 19:55:52,104 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family1/cb0b84fb6a4c49e5b9412efabc8ed725 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family1/cb0b84fb6a4c49e5b9412efabc8ed725 2023-03-08 19:55:52,467 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202303082fe6ad3fb83e45aa8f01bbfb4d6d6de6_del 2023-03-08 19:55:52,468 INFO [Listener at localhost/34049] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303082fe6ad3fb83e45aa8f01bbfb4d6d6de6_del to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202303082fe6ad3fb83e45aa8f01bbfb4d6d6de6_del 2023-03-08 19:55:53,282 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dfa32658f2544fb3b9bcfa546876ef19 2023-03-08 19:55:53,282 DEBUG [Listener at localhost/34049] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/.tmp/family2/dfa32658f2544fb3b9bcfa546876ef19 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family2/dfa32658f2544fb3b9bcfa546876ef19 2023-03-08 19:55:53,286 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dfa32658f2544fb3b9bcfa546876ef19 2023-03-08 19:55:53,287 INFO [Listener at localhost/34049] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 400484c718b5dfcfabf2e62d3d3e1140/family2 of 400484c718b5dfcfabf2e62d3d3e1140 into dfa32658f2544fb3b9bcfa546876ef19(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 19:55:53,287 DEBUG [Listener at localhost/34049] regionserver.HRegion(2289): Compaction status journal for 400484c718b5dfcfabf2e62d3d3e1140: 2023-03-08 19:55:53,287 DEBUG [Listener at localhost/34049] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 19:55:53,287 DEBUG [Listener at localhost/34049] regionserver.HStore(1912): 4f3474af7bec67a38b1529f33c13c7a6/family1 is initiating major compaction (all files) 2023-03-08 19:55:53,287 INFO [Listener at localhost/34049] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 19:55:53,287 INFO [Listener at localhost/34049] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 19:55:53,288 INFO [Listener at localhost/34049] regionserver.HRegion(2259): Starting compaction of 4f3474af7bec67a38b1529f33c13c7a6/family1 in ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6. 2023-03-08 19:55:53,288 INFO [Listener at localhost/34049] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family1/df669bc209bd4727824a56f1911e07d5, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family1/737ae079e14c4549b973fdbab6b5dc11, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family1/38146755cbe748c187b8021a02c70dde, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family1/aa7a1846a5944a5e965959b94b915c5e, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family1/68ccebb17c4e4533a14934d301b006ff] into tmpdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/.tmp, totalSize=26.6 K 2023-03-08 19:55:53,288 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting df669bc209bd4727824a56f1911e07d5, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678305340129 2023-03-08 19:55:53,289 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 737ae079e14c4549b973fdbab6b5dc11, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678305341908 2023-03-08 19:55:53,289 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 38146755cbe748c187b8021a02c70dde, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678305343306 2023-03-08 19:55:53,289 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting aa7a1846a5944a5e965959b94b915c5e, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678305345098 2023-03-08 19:55:53,290 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 68ccebb17c4e4533a14934d301b006ff, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 19:55:53,297 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family2/c6024a067fce4e1d92bbac35b4e28e0b, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family2/f071f8fcf7d5486590dcfdbc1b759bbc, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family2/641ff8bc83ea420ab9efa11895eb599b, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family2/2757b342b5f745aaa6edd352f5f6e416, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family2/09b77a2d02c74eeb9c815270281a9bb7] to archive 2023-03-08 19:55:53,298 INFO [Listener at localhost/34049] throttle.PressureAwareThroughputController(145): 4f3474af7bec67a38b1529f33c13c7a6#family1#compaction#154 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 19:55:53,298 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:55:53,300 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family2/c6024a067fce4e1d92bbac35b4e28e0b to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family2/c6024a067fce4e1d92bbac35b4e28e0b 2023-03-08 19:55:53,302 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family2/f071f8fcf7d5486590dcfdbc1b759bbc to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family2/f071f8fcf7d5486590dcfdbc1b759bbc 2023-03-08 19:55:53,303 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family2/641ff8bc83ea420ab9efa11895eb599b to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family2/641ff8bc83ea420ab9efa11895eb599b 2023-03-08 19:55:53,304 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family2/2757b342b5f745aaa6edd352f5f6e416 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family2/2757b342b5f745aaa6edd352f5f6e416 2023-03-08 19:55:53,306 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family2/09b77a2d02c74eeb9c815270281a9bb7 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family2/09b77a2d02c74eeb9c815270281a9bb7 2023-03-08 19:55:53,309 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230308879f55d34d3c4b08a5a6cbec97c812b3_del 2023-03-08 19:55:53,310 INFO [Listener at localhost/34049] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308879f55d34d3c4b08a5a6cbec97c812b3_del to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230308879f55d34d3c4b08a5a6cbec97c812b3_del 2023-03-08 19:55:54,123 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 72eb7fe1face48259a21d6a2aec1e4be 2023-03-08 19:55:54,123 DEBUG [Listener at localhost/34049] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/.tmp/family1/72eb7fe1face48259a21d6a2aec1e4be as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family1/72eb7fe1face48259a21d6a2aec1e4be 2023-03-08 19:55:54,127 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 72eb7fe1face48259a21d6a2aec1e4be 2023-03-08 19:55:54,128 INFO [Listener at localhost/34049] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 4f3474af7bec67a38b1529f33c13c7a6/family1 of 4f3474af7bec67a38b1529f33c13c7a6 into 72eb7fe1face48259a21d6a2aec1e4be(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 19:55:54,128 DEBUG [Listener at localhost/34049] regionserver.HRegion(2289): Compaction status journal for 4f3474af7bec67a38b1529f33c13c7a6: 2023-03-08 19:55:54,128 DEBUG [Listener at localhost/34049] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 19:55:54,128 DEBUG [Listener at localhost/34049] regionserver.HStore(1912): 4f3474af7bec67a38b1529f33c13c7a6/family2 is initiating major compaction (all files) 2023-03-08 19:55:54,128 INFO [Listener at localhost/34049] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 19:55:54,128 INFO [Listener at localhost/34049] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 19:55:54,130 INFO [Listener at localhost/34049] regionserver.HRegion(2259): Starting compaction of 4f3474af7bec67a38b1529f33c13c7a6/family2 in ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6. 2023-03-08 19:55:54,130 INFO [Listener at localhost/34049] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family2/6eebe7cea5834c5c8077435b56e8286e, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family2/fe4c4767549b45dc876c04af7a545a63, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family2/b29fa78b7d5948f89aa61ce292c34b8d, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family2/aa2ed92083484cff9293d5453ddbe91b, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family2/7c289eda1e664923ade347b489cf449c] into tmpdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/.tmp, totalSize=25.3 K 2023-03-08 19:55:54,130 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 6eebe7cea5834c5c8077435b56e8286e, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678305340129 2023-03-08 19:55:54,130 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting fe4c4767549b45dc876c04af7a545a63, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678305341908 2023-03-08 19:55:54,131 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting b29fa78b7d5948f89aa61ce292c34b8d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678305343306 2023-03-08 19:55:54,131 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting aa2ed92083484cff9293d5453ddbe91b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678305345098 2023-03-08 19:55:54,131 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 7c289eda1e664923ade347b489cf449c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 19:55:54,145 INFO [Listener at localhost/34049] throttle.PressureAwareThroughputController(145): 4f3474af7bec67a38b1529f33c13c7a6#family2#compaction#155 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 19:55:54,198 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family1/df669bc209bd4727824a56f1911e07d5, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family1/737ae079e14c4549b973fdbab6b5dc11, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family1/38146755cbe748c187b8021a02c70dde, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family1/aa7a1846a5944a5e965959b94b915c5e, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family1/68ccebb17c4e4533a14934d301b006ff] to archive 2023-03-08 19:55:54,199 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:55:54,201 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family1/df669bc209bd4727824a56f1911e07d5 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family1/df669bc209bd4727824a56f1911e07d5 2023-03-08 19:55:54,202 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family1/737ae079e14c4549b973fdbab6b5dc11 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family1/737ae079e14c4549b973fdbab6b5dc11 2023-03-08 19:55:54,203 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family1/38146755cbe748c187b8021a02c70dde to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family1/38146755cbe748c187b8021a02c70dde 2023-03-08 19:55:54,204 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family1/aa7a1846a5944a5e965959b94b915c5e to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family1/aa7a1846a5944a5e965959b94b915c5e 2023-03-08 19:55:54,205 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family1/68ccebb17c4e4533a14934d301b006ff to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family1/68ccebb17c4e4533a14934d301b006ff 2023-03-08 19:55:54,553 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023030853d10cd27c0e4ea9a3951da6f031364c_del 2023-03-08 19:55:54,553 INFO [Listener at localhost/34049] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023030853d10cd27c0e4ea9a3951da6f031364c_del to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c2023030853d10cd27c0e4ea9a3951da6f031364c_del 2023-03-08 19:55:54,968 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bb55fc0a6f6c4ac5ae00061aa7847e22 2023-03-08 19:55:54,968 DEBUG [Listener at localhost/34049] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/.tmp/family2/bb55fc0a6f6c4ac5ae00061aa7847e22 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family2/bb55fc0a6f6c4ac5ae00061aa7847e22 2023-03-08 19:55:54,971 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bb55fc0a6f6c4ac5ae00061aa7847e22 2023-03-08 19:55:54,972 INFO [Listener at localhost/34049] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 4f3474af7bec67a38b1529f33c13c7a6/family2 of 4f3474af7bec67a38b1529f33c13c7a6 into bb55fc0a6f6c4ac5ae00061aa7847e22(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 19:55:54,972 DEBUG [Listener at localhost/34049] regionserver.HRegion(2289): Compaction status journal for 4f3474af7bec67a38b1529f33c13c7a6: 2023-03-08 19:55:54,975 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 19:55:54,978 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 19:55:54,981 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 19:55:54,982 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 19:55:54,986 INFO [Listener at localhost/34049] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-08 19:55:54,998 INFO [Listener at localhost/34049] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 3 del files, 3 selected files, and 0 irrelevant files table 'ns:testMinorCompaction' and column 'family1' 2023-03-08 19:55:55,000 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family2/6eebe7cea5834c5c8077435b56e8286e, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family2/fe4c4767549b45dc876c04af7a545a63, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family2/b29fa78b7d5948f89aa61ce292c34b8d, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family2/aa2ed92083484cff9293d5453ddbe91b, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family2/7c289eda1e664923ade347b489cf449c] to archive 2023-03-08 19:55:55,001 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:55:55,001 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230308f0465298ca354f6c91462fc234440c76_del 2023-03-08 19:55:55,002 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family2/6eebe7cea5834c5c8077435b56e8286e to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family2/6eebe7cea5834c5c8077435b56e8286e 2023-03-08 19:55:55,003 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family2/fe4c4767549b45dc876c04af7a545a63 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family2/fe4c4767549b45dc876c04af7a545a63 2023-03-08 19:55:55,005 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family2/b29fa78b7d5948f89aa61ce292c34b8d to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family2/b29fa78b7d5948f89aa61ce292c34b8d 2023-03-08 19:55:55,005 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202303084620368a7a13468193cde834527f10c5_del 2023-03-08 19:55:55,006 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family2/aa2ed92083484cff9293d5453ddbe91b to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family2/aa2ed92083484cff9293d5453ddbe91b 2023-03-08 19:55:55,007 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family2/7c289eda1e664923ade347b489cf449c to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family2/7c289eda1e664923ade347b489cf449c 2023-03-08 19:55:55,008 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230308879f55d34d3c4b08a5a6cbec97c812b3_del 2023-03-08 19:55:55,008 INFO [Listener at localhost/34049] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='ns:testMinorCompaction' column='family1' 2023-03-08 19:55:55,009 DEBUG [Listener at localhost/34049] zookeeper.ReadOnlyZKClient(139): Connect 0x1b1d8602 to 127.0.0.1:56751 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 19:55:55,014 DEBUG [Listener at localhost/34049] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@424829e2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 19:55:55,015 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230308 for table='ns:testMinorCompaction' and column='family1' 2023-03-08 19:55:55,015 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230308 for table='ns:testMinorCompaction' and column='family1' 2023-03-08 19:55:55,015 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230308 for table='ns:testMinorCompaction' and column='family1' 2023-03-08 19:55:55,037 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308e0920439b84c4047b6adc6dce5951ef7 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230308e0920439b84c4047b6adc6dce5951ef7 2023-03-08 19:55:55,037 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230308 table='ns:testMinorCompaction' column='family1' 2023-03-08 19:55:55,050 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 19:55:55,053 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:32842, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 19:55:55,060 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 19:55:55,063 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54420, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 19:55:55,077 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230308/family1/1b0d3d1bc1454501925416ac9a461d84 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x01] 2023-03-08 19:55:55,078 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6., hostname=jenkins-hbase4.apache.org,33367,1678305218680, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230308/family1/1b0d3d1bc1454501925416ac9a461d84}] 2023-03-08 19:55:55,086 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HStore(763): Validating hfile at hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230308/family1/1b0d3d1bc1454501925416ac9a461d84 for inclusion in 4f3474af7bec67a38b1529f33c13c7a6/family1 2023-03-08 19:55:55,088 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x01 2023-03-08 19:55:55,088 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-08 19:55:55,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HRegion(2446): Flush status journal for 4f3474af7bec67a38b1529f33c13c7a6: 2023-03-08 19:55:55,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230308/family1/1b0d3d1bc1454501925416ac9a461d84 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__ns_testMinorCompaction__dmouhkdugn0652kqmhe3sogdemk57ne6j892tbh3d29latq1f657qv5sa8d8venh/family1/1b0d3d1bc1454501925416ac9a461d84 2023-03-08 19:55:55,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__ns_testMinorCompaction__dmouhkdugn0652kqmhe3sogdemk57ne6j892tbh3d29latq1f657qv5sa8d8venh/family1/1b0d3d1bc1454501925416ac9a461d84 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family1/15594eeafef24e10a3bbb538682674c9_SeqId_28_ 2023-03-08 19:55:55,091 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HStore(849): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__ns_testMinorCompaction__dmouhkdugn0652kqmhe3sogdemk57ne6j892tbh3d29latq1f657qv5sa8d8venh/family1/1b0d3d1bc1454501925416ac9a461d84 into 4f3474af7bec67a38b1529f33c13c7a6/family1 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family1/15594eeafef24e10a3bbb538682674c9_SeqId_28_ - updating store file list. 2023-03-08 19:55:55,094 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HStore(882): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family1/15594eeafef24e10a3bbb538682674c9_SeqId_28_ into 4f3474af7bec67a38b1529f33c13c7a6/family1 2023-03-08 19:55:55,094 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HStore(855): Successfully loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__ns_testMinorCompaction__dmouhkdugn0652kqmhe3sogdemk57ne6j892tbh3d29latq1f657qv5sa8d8venh/family1/1b0d3d1bc1454501925416ac9a461d84 into 4f3474af7bec67a38b1529f33c13c7a6/family1 (new location: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/family1/15594eeafef24e10a3bbb538682674c9_SeqId_28_) 2023-03-08 19:55:55,095 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__ns_testMinorCompaction__dmouhkdugn0652kqmhe3sogdemk57ne6j892tbh3d29latq1f657qv5sa8d8venh/family1/1b0d3d1bc1454501925416ac9a461d84 2023-03-08 19:55:55,097 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230308 table='ns:testMinorCompaction' column='family1' 2023-03-08 19:55:55,097 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:55:55,098 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202303088305dee3f3864f32b14bffe13cf965fe to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202303088305dee3f3864f32b14bffe13cf965fe 2023-03-08 19:55:55,098 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-03-08 19:55:55,098 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230308' 2023-03-08 19:55:55,099 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230308 for table='ns:testMinorCompaction' and column='family1' 2023-03-08 19:55:55,840 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308ea8b7ca1306747a5abae9935af320298 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230308ea8b7ca1306747a5abae9935af320298 2023-03-08 19:55:55,840 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308ffb2c3b94f594a669228139d3542d02c to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230308ffb2c3b94f594a669228139d3542d02c 2023-03-08 19:55:55,841 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230308 table='ns:testMinorCompaction' column='family1' 2023-03-08 19:55:55,841 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230308 table='ns:testMinorCompaction' column='family1' 2023-03-08 19:55:55,867 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230308/family1/088362fb4daf482aa3821285749cb9f3 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x01] 2023-03-08 19:55:55,869 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140., hostname=jenkins-hbase4.apache.org,33367,1678305218680, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230308/family1/088362fb4daf482aa3821285749cb9f3}] 2023-03-08 19:55:55,871 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230308/family1/1c67c67d242147f5817faa74e762dad7 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x01] 2023-03-08 19:55:55,871 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4., hostname=jenkins-hbase4.apache.org,33367,1678305218680, seqNum=2 for row with hfile group [{family1,hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230308/family1/1c67c67d242147f5817faa74e762dad7}] 2023-03-08 19:55:55,871 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(763): Validating hfile at hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230308/family1/088362fb4daf482aa3821285749cb9f3 for inclusion in 400484c718b5dfcfabf2e62d3d3e1140/family1 2023-03-08 19:55:55,873 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HStore(763): Validating hfile at hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230308/family1/1c67c67d242147f5817faa74e762dad7 for inclusion in 246f7af23020ff6b2e393b5a81b2e3c4/family1 2023-03-08 19:55:55,874 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x01 2023-03-08 19:55:55,874 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-08 19:55:55,874 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(2446): Flush status journal for 400484c718b5dfcfabf2e62d3d3e1140: 2023-03-08 19:55:55,875 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x01 2023-03-08 19:55:55,875 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HStore(777): Region bounds: first= last=1 2023-03-08 19:55:55,875 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230308/family1/088362fb4daf482aa3821285749cb9f3 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__ns_testMinorCompaction__u2d7s03aj3qlc0jgi3k7rjkkha4m2k5entdu9ibq97jk1tra53neuntf3lqonsfn/family1/088362fb4daf482aa3821285749cb9f3 2023-03-08 19:55:55,875 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HRegion(2446): Flush status journal for 246f7af23020ff6b2e393b5a81b2e3c4: 2023-03-08 19:55:55,876 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230308/family1/1c67c67d242147f5817faa74e762dad7 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__ns_testMinorCompaction__lb8ecc9tmgtcr0600e5g1bc8qc6obs4et1th6c26gn96bnl8u9coj6csm5s1irgi/family1/1c67c67d242147f5817faa74e762dad7 2023-03-08 19:55:55,880 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__ns_testMinorCompaction__u2d7s03aj3qlc0jgi3k7rjkkha4m2k5entdu9ibq97jk1tra53neuntf3lqonsfn/family1/088362fb4daf482aa3821285749cb9f3 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family1/c05130c79c8947fdb78775857a732224_SeqId_28_ 2023-03-08 19:55:55,880 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(849): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__ns_testMinorCompaction__u2d7s03aj3qlc0jgi3k7rjkkha4m2k5entdu9ibq97jk1tra53neuntf3lqonsfn/family1/088362fb4daf482aa3821285749cb9f3 into 400484c718b5dfcfabf2e62d3d3e1140/family1 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family1/c05130c79c8947fdb78775857a732224_SeqId_28_ - updating store file list. 2023-03-08 19:55:55,881 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__ns_testMinorCompaction__lb8ecc9tmgtcr0600e5g1bc8qc6obs4et1th6c26gn96bnl8u9coj6csm5s1irgi/family1/1c67c67d242147f5817faa74e762dad7 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family1/1688ee5499ae49dc87ba7caa15728e87_SeqId_28_ 2023-03-08 19:55:55,881 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HStore(849): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__ns_testMinorCompaction__lb8ecc9tmgtcr0600e5g1bc8qc6obs4et1th6c26gn96bnl8u9coj6csm5s1irgi/family1/1c67c67d242147f5817faa74e762dad7 into 246f7af23020ff6b2e393b5a81b2e3c4/family1 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family1/1688ee5499ae49dc87ba7caa15728e87_SeqId_28_ - updating store file list. 2023-03-08 19:55:55,884 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(882): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family1/c05130c79c8947fdb78775857a732224_SeqId_28_ into 400484c718b5dfcfabf2e62d3d3e1140/family1 2023-03-08 19:55:55,884 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(855): Successfully loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__ns_testMinorCompaction__u2d7s03aj3qlc0jgi3k7rjkkha4m2k5entdu9ibq97jk1tra53neuntf3lqonsfn/family1/088362fb4daf482aa3821285749cb9f3 into 400484c718b5dfcfabf2e62d3d3e1140/family1 (new location: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/family1/c05130c79c8947fdb78775857a732224_SeqId_28_) 2023-03-08 19:55:55,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__ns_testMinorCompaction__u2d7s03aj3qlc0jgi3k7rjkkha4m2k5entdu9ibq97jk1tra53neuntf3lqonsfn/family1/088362fb4daf482aa3821285749cb9f3 2023-03-08 19:55:55,884 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HStore(882): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family1/1688ee5499ae49dc87ba7caa15728e87_SeqId_28_ into 246f7af23020ff6b2e393b5a81b2e3c4/family1 2023-03-08 19:55:55,884 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.HStore(855): Successfully loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__ns_testMinorCompaction__lb8ecc9tmgtcr0600e5g1bc8qc6obs4et1th6c26gn96bnl8u9coj6csm5s1irgi/family1/1c67c67d242147f5817faa74e762dad7 into 246f7af23020ff6b2e393b5a81b2e3c4/family1 (new location: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/family1/1688ee5499ae49dc87ba7caa15728e87_SeqId_28_) 2023-03-08 19:55:55,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__ns_testMinorCompaction__lb8ecc9tmgtcr0600e5g1bc8qc6obs4et1th6c26gn96bnl8u9coj6csm5s1irgi/family1/1c67c67d242147f5817faa74e762dad7 2023-03-08 19:55:55,886 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230308 table='ns:testMinorCompaction' column='family1' 2023-03-08 19:55:55,886 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230308 table='ns:testMinorCompaction' column='family1' 2023-03-08 19:55:55,886 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:55:55,887 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:55:55,888 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303084a3646332df14691ac2039b01851dcad to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303084a3646332df14691ac2039b01851dcad 2023-03-08 19:55:55,888 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230308a10ad2f7870249df9a0e3e77cff902d3 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230308a10ad2f7870249df9a0e3e77cff902d3 2023-03-08 19:55:55,888 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-03-08 19:55:55,888 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-03-08 19:55:55,888 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230308' 2023-03-08 19:55:55,888 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230308' 2023-03-08 19:55:55,888 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230308 for table='ns:testMinorCompaction' and column='family1' 2023-03-08 19:55:55,888 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230308 for table='ns:testMinorCompaction' and column='family1' 2023-03-08 19:55:55,889 INFO [Listener at localhost/34049] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='ns:testMinorCompaction' column='family1' 2023-03-08 19:55:55,891 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 19:55:55,892 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 19:55:55,895 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 19:55:55,896 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 19:55:55,915 INFO [Listener at localhost/34049] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompaction Thread=248 (was 285), OpenFileDescriptor=625 (was 759), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=137 (was 152), ProcessCount=170 (was 170), AvailableMemoryMB=8199 (was 8216) 2023-03-08 19:55:55,927 INFO [Listener at localhost/34049] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=248, OpenFileDescriptor=625, MaxFileDescriptor=60000, SystemLoadAverage=137, ProcessCount=170, AvailableMemoryMB=8198 2023-03-08 19:55:55,929 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-08 19:55:55,930 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:55,931 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 19:55:55,932 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithMonthlyPolicy" procId is: 112 2023-03-08 19:55:55,932 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 19:55:55,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-08 19:55:55,933 DEBUG [HFileArchiver-19] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28 2023-03-08 19:55:55,934 DEBUG [HFileArchiver-19] backup.HFileArchiver(153): Directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28 empty. 2023-03-08 19:55:55,934 DEBUG [HFileArchiver-19] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28 2023-03-08 19:55:55,934 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithMonthlyPolicy regions 2023-03-08 19:55:56,033 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-08 19:55:56,234 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-08 19:55:56,341 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-08 19:55:56,342 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => b1a5a8585f5b1218e278f2f033438f28, NAME => 'testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp 2023-03-08 19:55:56,346 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:55:56,347 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1604): Closing b1a5a8585f5b1218e278f2f033438f28, disabling compactions & flushes 2023-03-08 19:55:56,347 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. 2023-03-08 19:55:56,347 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. 2023-03-08 19:55:56,347 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. after waiting 0 ms 2023-03-08 19:55:56,347 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. 2023-03-08 19:55:56,347 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1838): Closed testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. 2023-03-08 19:55:56,347 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for b1a5a8585f5b1218e278f2f033438f28: 2023-03-08 19:55:56,348 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 19:55:56,349 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678305356349"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678305356349"}]},"ts":"1678305356349"} 2023-03-08 19:55:56,350 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-08 19:55:56,351 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 19:55:56,351 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678305356351"}]},"ts":"1678305356351"} 2023-03-08 19:55:56,352 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-03-08 19:55:56,355 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=b1a5a8585f5b1218e278f2f033438f28, ASSIGN}] 2023-03-08 19:55:56,356 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=b1a5a8585f5b1218e278f2f033438f28, ASSIGN 2023-03-08 19:55:56,356 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=b1a5a8585f5b1218e278f2f033438f28, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33367,1678305218680; forceNewPlan=false, retain=false 2023-03-08 19:55:56,507 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=b1a5a8585f5b1218e278f2f033438f28, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:56,508 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678305356507"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678305356507"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678305356507"}]},"ts":"1678305356507"} 2023-03-08 19:55:56,509 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; OpenRegionProcedure b1a5a8585f5b1218e278f2f033438f28, server=jenkins-hbase4.apache.org,33367,1678305218680}] 2023-03-08 19:55:56,535 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-08 19:55:56,663 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. 2023-03-08 19:55:56,664 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b1a5a8585f5b1218e278f2f033438f28, NAME => 'testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28.', STARTKEY => '', ENDKEY => ''} 2023-03-08 19:55:56,664 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithMonthlyPolicy b1a5a8585f5b1218e278f2f033438f28 2023-03-08 19:55:56,664 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:55:56,664 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b1a5a8585f5b1218e278f2f033438f28 2023-03-08 19:55:56,664 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b1a5a8585f5b1218e278f2f033438f28 2023-03-08 19:55:56,665 INFO [StoreOpener-b1a5a8585f5b1218e278f2f033438f28-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region b1a5a8585f5b1218e278f2f033438f28 2023-03-08 19:55:56,666 DEBUG [StoreOpener-b1a5a8585f5b1218e278f2f033438f28-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1 2023-03-08 19:55:56,666 DEBUG [StoreOpener-b1a5a8585f5b1218e278f2f033438f28-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1 2023-03-08 19:55:56,667 INFO [StoreOpener-b1a5a8585f5b1218e278f2f033438f28-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b1a5a8585f5b1218e278f2f033438f28 columnFamilyName family1 2023-03-08 19:55:56,668 INFO [StoreOpener-b1a5a8585f5b1218e278f2f033438f28-1] regionserver.HStore(310): Store=b1a5a8585f5b1218e278f2f033438f28/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:55:56,669 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28 2023-03-08 19:55:56,669 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28 2023-03-08 19:55:56,671 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for b1a5a8585f5b1218e278f2f033438f28 2023-03-08 19:55:56,672 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 19:55:56,673 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b1a5a8585f5b1218e278f2f033438f28; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65428919, jitterRate=-0.0250331312417984}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-08 19:55:56,673 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b1a5a8585f5b1218e278f2f033438f28: 2023-03-08 19:55:56,673 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28., pid=114, masterSystemTime=1678305356661 2023-03-08 19:55:56,674 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. 2023-03-08 19:55:56,675 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. 2023-03-08 19:55:56,675 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=b1a5a8585f5b1218e278f2f033438f28, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:56,675 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678305356675"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678305356675"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678305356675"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678305356675"}]},"ts":"1678305356675"} 2023-03-08 19:55:56,677 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-03-08 19:55:56,677 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; OpenRegionProcedure b1a5a8585f5b1218e278f2f033438f28, server=jenkins-hbase4.apache.org,33367,1678305218680 in 167 msec 2023-03-08 19:55:56,678 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-03-08 19:55:56,678 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=b1a5a8585f5b1218e278f2f033438f28, ASSIGN in 323 msec 2023-03-08 19:55:56,678 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 19:55:56,679 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678305356679"}]},"ts":"1678305356679"} 2023-03-08 19:55:56,679 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-03-08 19:55:56,682 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 19:55:56,683 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy in 753 msec 2023-03-08 19:55:57,036 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-08 19:55:57,037 INFO [Listener at localhost/34049] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithMonthlyPolicy, procId: 112 completed 2023-03-08 19:55:57,060 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:55:57,062 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:57,062 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 19:55:57,062 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:55:57,062 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 19:55:57,062 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 19:55:57,062 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:57,062 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:57,064 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:57,064 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:55:57,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:55:57,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:55:57,065 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:57,065 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:55:57,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:57,065 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:57,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:55:57,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:57,065 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:57,076 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:57,076 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 19:55:57,076 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:55:57,076 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 19:55:57,076 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:55:57,076 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:55:57,076 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. 2023-03-08 19:55:57,076 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. started... 2023-03-08 19:55:57,077 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b1a5a8585f5b1218e278f2f033438f28 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-08 19:55:57,097 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511302192fc08b4b64ea2905586cc213b98bd to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511302192fc08b4b64ea2905586cc213b98bd 2023-03-08 19:55:57,502 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/.tmp/family1/b4560772cc64412d807e338712b29416 2023-03-08 19:55:57,505 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/.tmp/family1/b4560772cc64412d807e338712b29416 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/b4560772cc64412d807e338712b29416 2023-03-08 19:55:57,509 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/b4560772cc64412d807e338712b29416, entries=1000, sequenceid=5, filesize=166.0 K 2023-03-08 19:55:57,510 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for b1a5a8585f5b1218e278f2f033438f28 in 433ms, sequenceid=5, compaction requested=false 2023-03-08 19:55:57,510 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 19:55:57,511 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b1a5a8585f5b1218e278f2f033438f28: 2023-03-08 19:55:57,511 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. 2023-03-08 19:55:57,511 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:55:57,511 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:55:57,511 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:57,511 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 19:55:57,511 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-08 19:55:57,512 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:57,512 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:57,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:57,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:57,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:57,513 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:57,513 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:55:57,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:57,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:57,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:57,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:57,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:57,514 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 19:55:57,514 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@32881363[Count = 0] remaining members to acquire global barrier 2023-03-08 19:55:57,514 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 19:55:57,514 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:57,515 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:57,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:57,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:57,515 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:57,515 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:55:57,515 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 19:55:57,515 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-08 19:55:57,515 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:55:57,517 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 19:55:57,517 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:57,517 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:55:57,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:57,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:57,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:57,517 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-08 19:55:57,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:57,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:57,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:57,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:57,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:57,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:57,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:57,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:55:57,519 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 19:55:57,519 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-08 19:55:57,519 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:55:57,519 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:55:57,519 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:57,520 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:55:57,521 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:57,521 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:57,521 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:55:57,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:57,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:57,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:57,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:57,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:57,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:57,521 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:57,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:55:57,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:55:57,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:57,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:57,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:57,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:57,522 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:57,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:57,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:57,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:57,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:57,524 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:57,524 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:55:57,524 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:57,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:55:57,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:55:57,524 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:55:57,524 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 19:55:57,525 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:55:57,524 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:57,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:55:57,524 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:55:57,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:55:57,525 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:57,525 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:57,525 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 19:55:57,525 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:55:57,525 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:57,525 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:57,625 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:55:57,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:55:57,630 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:55:57,632 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 19:55:57,632 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:57,632 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 19:55:57,632 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:55:57,632 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 19:55:57,632 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 19:55:57,633 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:57,633 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:57,634 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:57,634 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:55:57,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:55:57,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:55:57,634 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:57,634 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:55:57,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:57,635 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:57,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:55:57,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:57,635 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:57,635 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 19:55:57,635 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:57,635 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 19:55:57,635 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:55:57,635 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 19:55:57,635 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:55:57,635 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:55:57,636 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. 2023-03-08 19:55:57,636 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. started... 2023-03-08 19:55:57,636 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b1a5a8585f5b1218e278f2f033438f28 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-08 19:55:58,046 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128ee847ffd5df7493e8831556bc39f3838 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128ee847ffd5df7493e8831556bc39f3838 2023-03-08 19:55:58,451 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/.tmp/family1/b1ea3450cbe64b5f825bcf72b5bf6bee 2023-03-08 19:55:58,455 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/.tmp/family1/b1ea3450cbe64b5f825bcf72b5bf6bee as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/b1ea3450cbe64b5f825bcf72b5bf6bee 2023-03-08 19:55:58,458 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/b1ea3450cbe64b5f825bcf72b5bf6bee, entries=1, sequenceid=9, filesize=5.0 K 2023-03-08 19:55:58,459 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for b1a5a8585f5b1218e278f2f033438f28 in 823ms, sequenceid=9, compaction requested=false 2023-03-08 19:55:58,459 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b1a5a8585f5b1218e278f2f033438f28: 2023-03-08 19:55:58,459 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. 2023-03-08 19:55:58,459 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:55:58,459 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:55:58,459 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:58,459 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 19:55:58,459 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-08 19:55:58,462 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:58,462 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:58,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:58,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:58,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:58,462 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:58,462 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:55:58,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:58,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:58,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:58,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:58,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:58,463 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 19:55:58,463 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2cd34a4a[Count = 0] remaining members to acquire global barrier 2023-03-08 19:55:58,463 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 19:55:58,464 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:58,465 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:58,465 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:58,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:58,465 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 19:55:58,465 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-08 19:55:58,465 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:55:58,465 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:58,465 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:55:58,466 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:58,466 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 19:55:58,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:58,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:58,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:58,466 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:55:58,467 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-08 19:55:58,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:58,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:58,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:58,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:58,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:58,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:58,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:58,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:55:58,470 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 19:55:58,470 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-08 19:55:58,470 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:55:58,470 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:55:58,470 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:58,470 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:55:58,471 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:58,471 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:58,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:58,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:58,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:58,471 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:55:58,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:58,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:58,472 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:58,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:55:58,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:58,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:55:58,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:58,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:58,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:58,473 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:58,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:58,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:58,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:58,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:58,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:58,476 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:58,477 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:58,477 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:58,477 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:55:58,477 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:55:58,477 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 19:55:58,477 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:55:58,477 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:58,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:55:58,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:55:58,477 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:55:58,477 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:58,477 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:58,477 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:58,477 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 19:55:58,477 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:55:58,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:55:58,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:55:58,477 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 19:55:58,577 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:55:58,578 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:55:58,582 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:55:58,584 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 19:55:58,584 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:58,584 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 19:55:58,584 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:55:58,584 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 19:55:58,584 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 19:55:58,584 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:58,584 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:58,589 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:55:58,589 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:58,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:55:58,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:55:58,590 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:58,590 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:55:58,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:58,590 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:58,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:55:58,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:58,591 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:58,591 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 19:55:58,591 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:58,591 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 19:55:58,591 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:55:58,591 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 19:55:58,591 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:55:58,591 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:55:58,591 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. 2023-03-08 19:55:58,591 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. started... 2023-03-08 19:55:58,592 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b1a5a8585f5b1218e278f2f033438f28 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:55:58,603 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201db5bde649d484533bc1326b4f5e1f43c to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201db5bde649d484533bc1326b4f5e1f43c 2023-03-08 19:55:59,008 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/.tmp/family1/ab62f17ff0404c589c0cf688ef2455ff 2023-03-08 19:55:59,012 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/.tmp/family1/ab62f17ff0404c589c0cf688ef2455ff as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/ab62f17ff0404c589c0cf688ef2455ff 2023-03-08 19:55:59,015 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/ab62f17ff0404c589c0cf688ef2455ff, entries=1, sequenceid=13, filesize=4.9 K 2023-03-08 19:55:59,015 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b1a5a8585f5b1218e278f2f033438f28 in 424ms, sequenceid=13, compaction requested=true 2023-03-08 19:55:59,015 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b1a5a8585f5b1218e278f2f033438f28: 2023-03-08 19:55:59,015 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. 2023-03-08 19:55:59,015 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:55:59,016 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:55:59,016 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:59,016 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 19:55:59,016 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-08 19:55:59,017 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:59,017 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:59,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:59,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:59,018 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,018 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:55:59,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:59,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:59,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:59,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:59,019 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 19:55:59,019 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@76af8cb0[Count = 0] remaining members to acquire global barrier 2023-03-08 19:55:59,019 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 19:55:59,019 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,021 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,022 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 19:55:59,022 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-08 19:55:59,022 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:55:59,022 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:59,022 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:55:59,023 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:59,023 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 19:55:59,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:59,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:59,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:59,023 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:55:59,024 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-08 19:55:59,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:59,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:59,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:59,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:59,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:59,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:55:59,025 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 19:55:59,025 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-08 19:55:59,026 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:55:59,026 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:55:59,026 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,026 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:55:59,027 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,027 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,027 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:55:59,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:59,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:59,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,028 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:59,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:55:59,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:59,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:55:59,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:59,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,029 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:59,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:59,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:59,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:59,031 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:59,031 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,031 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:55:59,031 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,031 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:55:59,031 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 19:55:59,031 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:55:59,031 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:59,031 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:55:59,031 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:55:59,031 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:55:59,031 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,031 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,031 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 19:55:59,031 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:55:59,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:55:59,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:55:59,132 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:55:59,133 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:55:59,136 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:55:59,138 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 19:55:59,138 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,138 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 19:55:59,138 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:55:59,138 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 19:55:59,138 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 19:55:59,139 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,139 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,141 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:59,141 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:55:59,141 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:55:59,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:55:59,141 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:59,142 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:55:59,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,142 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:55:59,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,142 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,142 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 19:55:59,142 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,143 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 19:55:59,143 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:55:59,143 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 19:55:59,143 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:55:59,143 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:55:59,143 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. 2023-03-08 19:55:59,143 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. started... 2023-03-08 19:55:59,143 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b1a5a8585f5b1218e278f2f033438f28 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:55:59,554 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205092b41f5a63340b4b41362d7cdfbe42f to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205092b41f5a63340b4b41362d7cdfbe42f 2023-03-08 19:55:59,959 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/.tmp/family1/6dcb2189f2a1443b9348cbd52bcad0be 2023-03-08 19:55:59,963 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/.tmp/family1/6dcb2189f2a1443b9348cbd52bcad0be as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/6dcb2189f2a1443b9348cbd52bcad0be 2023-03-08 19:55:59,966 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/6dcb2189f2a1443b9348cbd52bcad0be, entries=1, sequenceid=17, filesize=4.9 K 2023-03-08 19:55:59,967 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b1a5a8585f5b1218e278f2f033438f28 in 824ms, sequenceid=17, compaction requested=true 2023-03-08 19:55:59,967 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b1a5a8585f5b1218e278f2f033438f28: 2023-03-08 19:55:59,967 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. 2023-03-08 19:55:59,967 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:55:59,967 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:55:59,967 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:59,967 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 19:55:59,967 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-08 19:55:59,968 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:59,968 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:59,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:59,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:59,969 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,969 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:55:59,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:59,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:59,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:59,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:59,970 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 19:55:59,970 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@40244b85[Count = 0] remaining members to acquire global barrier 2023-03-08 19:55:59,970 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 19:55:59,970 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,971 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,971 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 19:55:59,971 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-08 19:55:59,971 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:55:59,972 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:59,972 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:55:59,973 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:59,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:59,973 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 19:55:59,973 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:55:59,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:59,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:59,973 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-08 19:55:59,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:59,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:59,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:59,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:59,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:59,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:55:59,975 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 19:55:59,975 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-08 19:55:59,975 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:55:59,975 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:55:59,975 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,975 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:55:59,978 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,978 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:55:59,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:55:59,978 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:55:59,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,978 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:59,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:55:59,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:55:59,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:55:59,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:55:59,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,979 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:59,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:59,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:55:59,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:59,981 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:59,981 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:55:59,981 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,981 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:55:59,981 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,981 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:55:59,981 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,981 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,981 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:55:59,981 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 19:55:59,981 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:55:59,981 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 19:55:59,982 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:55:59,981 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:55:59,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:55:59,982 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 19:55:59,982 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:55:59,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:55:59,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:56:00,082 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:56:00,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:56:00,086 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:56:00,088 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 19:56:00,088 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,088 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 19:56:00,088 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:56:00,088 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 19:56:00,088 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 19:56:00,088 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,089 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,090 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,090 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:56:00,090 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:56:00,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:56:00,090 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,090 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:56:00,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,090 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:56:00,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,091 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,091 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 19:56:00,091 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,091 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 19:56:00,091 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:56:00,091 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 19:56:00,091 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:56:00,091 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:56:00,091 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. 2023-03-08 19:56:00,091 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. started... 2023-03-08 19:56:00,091 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b1a5a8585f5b1218e278f2f033438f28 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:56:00,502 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512285d2435c2383147f69c5245480074a167 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512285d2435c2383147f69c5245480074a167 2023-03-08 19:56:00,511 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/.tmp/family1/6a4cbc7fab364ff991d5138d20b7633d 2023-03-08 19:56:00,514 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/.tmp/family1/6a4cbc7fab364ff991d5138d20b7633d as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/6a4cbc7fab364ff991d5138d20b7633d 2023-03-08 19:56:00,517 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/6a4cbc7fab364ff991d5138d20b7633d, entries=1, sequenceid=21, filesize=4.9 K 2023-03-08 19:56:00,518 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b1a5a8585f5b1218e278f2f033438f28 in 427ms, sequenceid=21, compaction requested=true 2023-03-08 19:56:00,518 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b1a5a8585f5b1218e278f2f033438f28: 2023-03-08 19:56:00,518 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. 2023-03-08 19:56:00,518 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:56:00,518 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:56:00,518 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,518 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 19:56:00,518 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-08 19:56:00,520 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,520 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:56:00,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:56:00,520 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,520 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:56:00,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:56:00,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:56:00,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:56:00,522 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 19:56:00,522 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@32c54be9[Count = 0] remaining members to acquire global barrier 2023-03-08 19:56:00,522 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 19:56:00,522 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,524 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,524 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 19:56:00,524 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-08 19:56:00,524 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:56:00,524 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,524 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:56:00,525 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,526 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 19:56:00,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:56:00,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:56:00,526 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:56:00,526 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-08 19:56:00,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:56:00,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:56:00,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:56:00,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:56:00,528 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 19:56:00,528 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-08 19:56:00,528 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:56:00,528 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:56:00,528 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,528 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:56:00,530 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,530 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,530 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:56:00,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:56:00,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:56:00,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:56:00,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:56:00,530 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:56:00,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:56:00,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,531 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:56:00,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,536 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,536 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:56:00,536 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:56:00,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:56:00,536 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:56:00,536 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:56:00,536 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 19:56:00,536 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:56:00,536 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,536 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,536 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,536 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,536 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:56:00,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:56:00,537 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 19:56:00,537 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 19:56:00,537 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:56:00,637 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:56:00,638 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:56:00,641 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:56:00,643 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 19:56:00,643 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,643 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 19:56:00,643 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:56:00,643 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 19:56:00,643 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 19:56:00,644 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,644 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,645 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,645 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:56:00,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:56:00,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:56:00,645 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,645 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:56:00,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,646 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:56:00,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,646 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,646 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 19:56:00,646 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,646 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 19:56:00,646 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:56:00,646 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 19:56:00,647 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:56:00,647 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:56:00,647 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. 2023-03-08 19:56:00,647 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. started... 2023-03-08 19:56:00,647 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b1a5a8585f5b1218e278f2f033438f28 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:56:00,656 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123149d94bab47fe487ebfd2353d976412e4 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015123149d94bab47fe487ebfd2353d976412e4 2023-03-08 19:56:00,661 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/.tmp/family1/cc5e9db77f1348b19f4454e070bd8e7d 2023-03-08 19:56:00,664 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/.tmp/family1/cc5e9db77f1348b19f4454e070bd8e7d as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/cc5e9db77f1348b19f4454e070bd8e7d 2023-03-08 19:56:00,667 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/cc5e9db77f1348b19f4454e070bd8e7d, entries=1, sequenceid=25, filesize=4.9 K 2023-03-08 19:56:00,668 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b1a5a8585f5b1218e278f2f033438f28 in 21ms, sequenceid=25, compaction requested=true 2023-03-08 19:56:00,668 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b1a5a8585f5b1218e278f2f033438f28: 2023-03-08 19:56:00,668 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. 2023-03-08 19:56:00,668 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:56:00,668 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:56:00,668 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,668 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 19:56:00,668 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-08 19:56:00,669 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,669 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:56:00,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:56:00,670 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,670 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:56:00,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:56:00,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:56:00,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:56:00,671 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 19:56:00,671 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4f2d6e1[Count = 0] remaining members to acquire global barrier 2023-03-08 19:56:00,671 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 19:56:00,671 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,673 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,674 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 19:56:00,674 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-08 19:56:00,674 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:56:00,674 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,674 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:56:00,675 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 19:56:00,675 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:56:00,675 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,675 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-08 19:56:00,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:56:00,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:56:00,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:56:00,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:56:00,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:56:00,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:56:00,677 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 19:56:00,678 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-08 19:56:00,678 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:56:00,678 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:56:00,678 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,678 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:56:00,679 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,679 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,679 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:56:00,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:56:00,679 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:56:00,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:56:00,680 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:56:00,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:56:00,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:56:00,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,681 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:56:00,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,682 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,682 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:56:00,682 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,682 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:56:00,682 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,682 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:56:00,683 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:56:00,683 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:56:00,683 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 19:56:00,683 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:56:00,683 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,683 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,683 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,683 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:56:00,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:56:00,683 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 19:56:00,683 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:56:00,783 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:56:00,784 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:56:00,788 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:56:00,790 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 19:56:00,790 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,790 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 19:56:00,790 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:56:00,790 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 19:56:00,790 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 19:56:00,791 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,791 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,793 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,793 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:56:00,793 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:56:00,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:56:00,793 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,793 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:56:00,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,794 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:56:00,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,794 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,794 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 19:56:00,794 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,794 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 19:56:00,794 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:56:00,794 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 19:56:00,795 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:56:00,795 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:56:00,795 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. 2023-03-08 19:56:00,795 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. started... 2023-03-08 19:56:00,795 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b1a5a8585f5b1218e278f2f033438f28 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:56:00,805 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101c9b20bc3900c4a348548105ed413e048 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160101c9b20bc3900c4a348548105ed413e048 2023-03-08 19:56:00,810 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/.tmp/family1/2901885445234e4aa3ffd1a5a8c72fe4 2023-03-08 19:56:00,813 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/.tmp/family1/2901885445234e4aa3ffd1a5a8c72fe4 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/2901885445234e4aa3ffd1a5a8c72fe4 2023-03-08 19:56:00,816 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/2901885445234e4aa3ffd1a5a8c72fe4, entries=1, sequenceid=29, filesize=4.9 K 2023-03-08 19:56:00,817 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b1a5a8585f5b1218e278f2f033438f28 in 22ms, sequenceid=29, compaction requested=true 2023-03-08 19:56:00,817 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b1a5a8585f5b1218e278f2f033438f28: 2023-03-08 19:56:00,817 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. 2023-03-08 19:56:00,817 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:56:00,817 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:56:00,817 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,817 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 19:56:00,817 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-08 19:56:00,818 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,818 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:56:00,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:56:00,819 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,819 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:56:00,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:56:00,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:56:00,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:56:00,820 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 19:56:00,820 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@e3200c3[Count = 0] remaining members to acquire global barrier 2023-03-08 19:56:00,820 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 19:56:00,820 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,821 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,821 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 19:56:00,822 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-08 19:56:00,822 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,822 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:56:00,822 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:56:00,823 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,823 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 19:56:00,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:56:00,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:56:00,823 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:56:00,823 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-08 19:56:00,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:56:00,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:56:00,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:56:00,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:56:00,825 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 19:56:00,825 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-08 19:56:00,825 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:56:00,825 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:56:00,825 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,825 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:56:00,828 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,828 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:56:00,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:56:00,828 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:56:00,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:56:00,829 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:56:00,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:56:00,829 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:56:00,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,830 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:56:00,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,831 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:56:00,831 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:56:00,832 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:56:00,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:56:00,832 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 19:56:00,832 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:56:00,832 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:56:00,832 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:56:00,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:56:00,832 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,832 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 19:56:00,832 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:56:00,832 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,832 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,832 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,832 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,932 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:56:00,933 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:56:00,937 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:56:00,938 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 19:56:00,938 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,938 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 19:56:00,938 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:56:00,939 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 19:56:00,939 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 19:56:00,939 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,939 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,940 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,940 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:56:00,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:56:00,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:56:00,940 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:00,940 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:56:00,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,941 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:56:00,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,941 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,941 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 19:56:00,941 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:00,941 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 19:56:00,941 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:56:00,941 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 19:56:00,942 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:56:00,942 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:56:00,942 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. 2023-03-08 19:56:00,942 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. started... 2023-03-08 19:56:00,942 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b1a5a8585f5b1218e278f2f033438f28 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:56:01,352 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010338bfe701ec0045ccba2fc035d724bbe7 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010338bfe701ec0045ccba2fc035d724bbe7 2023-03-08 19:56:01,758 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/.tmp/family1/5eb45d065a294ab4b8f2a4f0a614c905 2023-03-08 19:56:01,761 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/.tmp/family1/5eb45d065a294ab4b8f2a4f0a614c905 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/5eb45d065a294ab4b8f2a4f0a614c905 2023-03-08 19:56:01,765 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/5eb45d065a294ab4b8f2a4f0a614c905, entries=1, sequenceid=33, filesize=4.9 K 2023-03-08 19:56:01,766 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b1a5a8585f5b1218e278f2f033438f28 in 824ms, sequenceid=33, compaction requested=true 2023-03-08 19:56:01,766 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b1a5a8585f5b1218e278f2f033438f28: 2023-03-08 19:56:01,766 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. 2023-03-08 19:56:01,766 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:56:01,766 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:56:01,766 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:01,766 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 19:56:01,766 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-08 19:56:01,768 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:01,768 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:01,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:01,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:56:01,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:56:01,769 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:01,769 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:56:01,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:56:01,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:56:01,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:01,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:01,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:56:01,770 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 19:56:01,770 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6ca80ae3[Count = 0] remaining members to acquire global barrier 2023-03-08 19:56:01,770 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 19:56:01,770 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:01,772 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:01,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:01,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:01,772 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 19:56:01,772 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-08 19:56:01,772 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:56:01,772 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:01,772 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:56:01,773 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:01,773 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 19:56:01,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:01,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:56:01,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:56:01,773 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:56:01,774 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-08 19:56:01,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:56:01,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:56:01,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:01,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:01,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:56:01,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:01,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:01,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:56:01,776 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 19:56:01,776 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-08 19:56:01,776 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:56:01,776 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:56:01,776 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:01,776 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:56:01,778 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:01,778 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:01,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:01,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:56:01,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:56:01,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:01,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:01,778 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:56:01,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:56:01,778 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:01,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:56:01,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:56:01,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:01,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:01,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:56:01,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:01,779 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:01,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:01,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:56:01,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:01,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:01,786 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:01,786 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:01,786 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:56:01,786 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:56:01,786 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:56:01,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:56:01,786 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 19:56:01,786 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 19:56:01,786 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:56:01,786 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:01,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:56:01,786 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:01,786 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:01,786 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:01,786 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:01,786 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 19:56:01,787 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:56:01,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:56:01,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:56:01,887 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:56:01,888 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:56:01,891 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:56:01,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 19:56:01,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:01,893 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 19:56:01,893 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:56:01,893 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 19:56:01,893 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 19:56:01,893 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:01,893 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:01,895 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:01,895 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:56:01,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:56:01,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:56:01,895 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:01,895 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:56:01,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:01,896 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:01,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:56:01,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:01,896 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:01,896 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 19:56:01,896 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:01,896 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 19:56:01,896 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:56:01,896 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 19:56:01,896 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:56:01,896 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:56:01,896 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. 2023-03-08 19:56:01,897 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. started... 2023-03-08 19:56:01,897 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b1a5a8585f5b1218e278f2f033438f28 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:56:02,202 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 19:56:02,310 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907304e1d25beb440b08604d747c11706d7 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20150907304e1d25beb440b08604d747c11706d7 2023-03-08 19:56:02,318 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/.tmp/family1/e75d6b26392c448c97ca37026fe91afe 2023-03-08 19:56:02,322 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/.tmp/family1/e75d6b26392c448c97ca37026fe91afe as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/e75d6b26392c448c97ca37026fe91afe 2023-03-08 19:56:02,325 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/e75d6b26392c448c97ca37026fe91afe, entries=1, sequenceid=37, filesize=4.9 K 2023-03-08 19:56:02,325 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b1a5a8585f5b1218e278f2f033438f28 in 428ms, sequenceid=37, compaction requested=true 2023-03-08 19:56:02,325 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b1a5a8585f5b1218e278f2f033438f28: 2023-03-08 19:56:02,325 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. 2023-03-08 19:56:02,326 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:56:02,326 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:56:02,326 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:02,326 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 19:56:02,326 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-08 19:56:02,327 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:02,327 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:02,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:02,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:56:02,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:56:02,328 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:02,328 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:56:02,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:56:02,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:56:02,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:02,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:02,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:56:02,329 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 19:56:02,329 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@742a9e9c[Count = 0] remaining members to acquire global barrier 2023-03-08 19:56:02,329 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 19:56:02,329 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:02,331 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:02,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:02,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:02,331 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 19:56:02,331 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:02,331 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:56:02,331 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-08 19:56:02,331 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:56:02,333 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:02,333 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 19:56:02,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:02,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:56:02,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:56:02,333 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:56:02,333 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-08 19:56:02,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:56:02,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:56:02,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:02,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:02,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:56:02,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:02,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:02,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:56:02,335 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 19:56:02,335 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-08 19:56:02,335 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:56:02,335 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:56:02,335 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:02,335 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:56:02,336 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:02,336 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:02,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:02,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:56:02,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:56:02,336 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:56:02,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:02,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:02,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:56:02,337 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:02,337 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:56:02,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:56:02,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:02,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:02,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:56:02,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:02,338 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:02,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:02,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:56:02,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:02,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:02,339 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:02,339 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:56:02,339 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:02,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:56:02,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:56:02,339 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:02,339 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 19:56:02,340 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:56:02,339 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:56:02,339 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:56:02,340 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:02,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:56:02,340 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-08 19:56:02,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:56:02,340 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 19:56:02,340 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:02,340 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:56:02,340 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:02,340 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:02,440 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:56:02,441 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:56:02,445 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:56:02,446 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-08 19:56:02,446 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:02,446 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 19:56:02,447 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:56:02,447 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-08 19:56:02,447 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-08 19:56:02,447 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:02,447 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:02,449 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:02,449 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:56:02,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:56:02,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:56:02,449 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:02,449 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:56:02,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:02,450 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:02,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:56:02,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:02,450 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:02,450 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-08 19:56:02,450 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:02,451 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-08 19:56:02,451 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:56:02,451 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-08 19:56:02,451 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-08 19:56:02,451 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:56:02,451 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. 2023-03-08 19:56:02,451 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. started... 2023-03-08 19:56:02,451 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b1a5a8585f5b1218e278f2f033438f28 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-08 19:56:02,863 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120e3597676df8946e7bc7c07bdcbe0a24a to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120e3597676df8946e7bc7c07bdcbe0a24a 2023-03-08 19:56:03,268 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/.tmp/family1/7e3a5997519444b5925d1bc746cd1175 2023-03-08 19:56:03,272 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/.tmp/family1/7e3a5997519444b5925d1bc746cd1175 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/7e3a5997519444b5925d1bc746cd1175 2023-03-08 19:56:03,274 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/7e3a5997519444b5925d1bc746cd1175, entries=1, sequenceid=41, filesize=4.9 K 2023-03-08 19:56:03,275 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b1a5a8585f5b1218e278f2f033438f28 in 824ms, sequenceid=41, compaction requested=true 2023-03-08 19:56:03,275 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b1a5a8585f5b1218e278f2f033438f28: 2023-03-08 19:56:03,275 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. 2023-03-08 19:56:03,275 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-08 19:56:03,275 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-08 19:56:03,275 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:03,275 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-08 19:56:03,275 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-08 19:56:03,277 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:03,277 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:03,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:03,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:56:03,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:56:03,277 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:03,277 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:56:03,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:56:03,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:56:03,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:03,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:03,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:56:03,278 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-08 19:56:03,279 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@667d0b26[Count = 0] remaining members to acquire global barrier 2023-03-08 19:56:03,279 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-08 19:56:03,279 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:03,280 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:03,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:03,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:03,280 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-08 19:56:03,280 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-08 19:56:03,280 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:56:03,280 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:03,280 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:56:03,281 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-08 19:56:03,281 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:03,281 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:56:03,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:03,282 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-08 19:56:03,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:56:03,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:56:03,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:56:03,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:56:03,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:03,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:03,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:56:03,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:03,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:03,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:56:03,284 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-08 19:56:03,284 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-08 19:56:03,284 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:56:03,284 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:56:03,284 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:03,284 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:56:03,291 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:03,291 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:03,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:03,291 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:56:03,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:03,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:03,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:56:03,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:56:03,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:56:03,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:56:03,291 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:03,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:56:03,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:03,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:03,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:56:03,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:03,292 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:03,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:03,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:56:03,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:03,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:03,294 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:56:03,294 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:56:03,294 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:03,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:56:03,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:56:03,295 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:03,295 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-08 19:56:03,295 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:56:03,295 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:56:03,295 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:03,295 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:03,295 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:03,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:56:03,295 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:03,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:56:03,295 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:03,295 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-08 19:56:03,295 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:56:03,395 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:56:03,396 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:56:03,397 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39989] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMinorCompactionWithMonthlyPolicy 2023-03-08 19:56:03,399 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] procedure2.ProcedureExecutor(1029): Stored pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 19:56:03,399 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 19:56:03,400 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 19:56:03,400 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=115, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678305963400 2023-03-08 19:56:03,401 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-08 19:56:03,402 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMinorCompactionWithMonthlyPolicy' and column 'family1' 2023-03-08 19:56:03,402 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 19:56:03,402 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] zookeeper.ReadOnlyZKClient(139): Connect 0x0bf75cd7 to 127.0.0.1:56751 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 19:56:03,407 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4f269271, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 19:56:03,408 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 19:56:03,835 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010351917d4753794727b6fe1425df1fe3c6 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010351917d4753794727b6fe1425df1fe3c6 2023-03-08 19:56:03,836 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 19:56:03,855 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 19:56:03,857 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57488, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 19:56:03,863 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 19:56:03,864 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54424, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 19:56:03,872 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/1fa292c7761a412487f77204d30b5e7f first=Optional[r5] last=Optional[r6] 2023-03-08 19:56:03,873 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28., hostname=jenkins-hbase4.apache.org,33367,1678305218680, seqNum=2 for row with hfile group [{family1,hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/1fa292c7761a412487f77204d30b5e7f}] 2023-03-08 19:56:03,877 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(763): Validating hfile at hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/1fa292c7761a412487f77204d30b5e7f for inclusion in b1a5a8585f5b1218e278f2f033438f28/family1 2023-03-08 19:56:03,878 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-03-08 19:56:03,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 19:56:03,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(2446): Flush status journal for b1a5a8585f5b1218e278f2f033438f28: 2023-03-08 19:56:03,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/1fa292c7761a412487f77204d30b5e7f to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMinorCompactionWithMonthlyPolicy__r2fcictad5qhrdbdgd9uhoppp3l6mae96lspp4c3vi8oqud3mtdfim4gvdcrtjn3/family1/1fa292c7761a412487f77204d30b5e7f 2023-03-08 19:56:03,881 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMinorCompactionWithMonthlyPolicy__r2fcictad5qhrdbdgd9uhoppp3l6mae96lspp4c3vi8oqud3mtdfim4gvdcrtjn3/family1/1fa292c7761a412487f77204d30b5e7f as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/e374ff65a6c347f8a656b0b552677c63_SeqId_44_ 2023-03-08 19:56:03,881 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(849): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMinorCompactionWithMonthlyPolicy__r2fcictad5qhrdbdgd9uhoppp3l6mae96lspp4c3vi8oqud3mtdfim4gvdcrtjn3/family1/1fa292c7761a412487f77204d30b5e7f into b1a5a8585f5b1218e278f2f033438f28/family1 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/e374ff65a6c347f8a656b0b552677c63_SeqId_44_ - updating store file list. 2023-03-08 19:56:03,884 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(882): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/e374ff65a6c347f8a656b0b552677c63_SeqId_44_ into b1a5a8585f5b1218e278f2f033438f28/family1 2023-03-08 19:56:03,884 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(855): Successfully loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMinorCompactionWithMonthlyPolicy__r2fcictad5qhrdbdgd9uhoppp3l6mae96lspp4c3vi8oqud3mtdfim4gvdcrtjn3/family1/1fa292c7761a412487f77204d30b5e7f into b1a5a8585f5b1218e278f2f033438f28/family1 (new location: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/e374ff65a6c347f8a656b0b552677c63_SeqId_44_) 2023-03-08 19:56:03,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMinorCompactionWithMonthlyPolicy__r2fcictad5qhrdbdgd9uhoppp3l6mae96lspp4c3vi8oqud3mtdfim4gvdcrtjn3/family1/1fa292c7761a412487f77204d30b5e7f 2023-03-08 19:56:03,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-08 19:56:03,886 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-08 19:56:03,886 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(1912): b1a5a8585f5b1218e278f2f033438f28/family1 is initiating minor compaction 2023-03-08 19:56:03,886 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@787ca95c; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 19:56:03,886 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of b1a5a8585f5b1218e278f2f033438f28/family1 in testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. 2023-03-08 19:56:03,886 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/b1ea3450cbe64b5f825bcf72b5bf6bee, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/ab62f17ff0404c589c0cf688ef2455ff, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/6dcb2189f2a1443b9348cbd52bcad0be, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/6a4cbc7fab364ff991d5138d20b7633d, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/cc5e9db77f1348b19f4454e070bd8e7d, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/2901885445234e4aa3ffd1a5a8c72fe4, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/5eb45d065a294ab4b8f2a4f0a614c905, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/e75d6b26392c448c97ca37026fe91afe, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/7e3a5997519444b5925d1bc746cd1175, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/e374ff65a6c347f8a656b0b552677c63_SeqId_44_] into tmpdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/.tmp, totalSize=49.7 K 2023-03-08 19:56:03,887 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting b1ea3450cbe64b5f825bcf72b5bf6bee, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-08 19:56:03,887 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting ab62f17ff0404c589c0cf688ef2455ff, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-08 19:56:03,887 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 6dcb2189f2a1443b9348cbd52bcad0be, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-08 19:56:03,887 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 6a4cbc7fab364ff991d5138d20b7633d, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-08 19:56:03,887 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting cc5e9db77f1348b19f4454e070bd8e7d, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-08 19:56:03,887 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 2901885445234e4aa3ffd1a5a8c72fe4, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-08 19:56:03,887 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 5eb45d065a294ab4b8f2a4f0a614c905, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-08 19:56:03,887 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting e75d6b26392c448c97ca37026fe91afe, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-08 19:56:03,887 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting 7e3a5997519444b5925d1bc746cd1175, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-08 19:56:03,887 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting e374ff65a6c347f8a656b0b552677c63_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-08 19:56:03,888 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 19:56:03,889 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:56:03,890 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160101c9b20bc3900c4a348548105ed413e048 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160101c9b20bc3900c4a348548105ed413e048 2023-03-08 19:56:03,891 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010338bfe701ec0045ccba2fc035d724bbe7 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010338bfe701ec0045ccba2fc035d724bbe7 2023-03-08 19:56:03,891 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-03-08 19:56:03,891 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-03-08 19:56:03,891 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 19:56:03,892 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 19:56:03,919 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] throttle.PressureAwareThroughputController(145): b1a5a8585f5b1218e278f2f033438f28#family1#compaction#166 average throughput is 0.23 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 19:56:04,343 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/.tmp/family1/e8acbef22de349189405241d63e6a1f0 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/e8acbef22de349189405241d63e6a1f0 2023-03-08 19:56:04,345 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231ca149c8dad46489a856967a3f90643f6 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231ca149c8dad46489a856967a3f90643f6 2023-03-08 19:56:04,345 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 19:56:04,347 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in b1a5a8585f5b1218e278f2f033438f28/family1 of b1a5a8585f5b1218e278f2f033438f28 into e8acbef22de349189405241d63e6a1f0(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 19:56:04,347 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for b1a5a8585f5b1218e278f2f033438f28: 2023-03-08 19:56:04,348 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1678305363886; duration=0sec 2023-03-08 19:56:04,348 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 19:56:04,374 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/a0d713d9a8a741379b3b30e8e0438c67 first=Optional[r1] last=Optional[r4] 2023-03-08 19:56:04,374 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28., hostname=jenkins-hbase4.apache.org,33367,1678305218680, seqNum=2 for row with hfile group [{family1,hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/a0d713d9a8a741379b3b30e8e0438c67}] 2023-03-08 19:56:04,377 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(763): Validating hfile at hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/a0d713d9a8a741379b3b30e8e0438c67 for inclusion in b1a5a8585f5b1218e278f2f033438f28/family1 2023-03-08 19:56:04,379 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-03-08 19:56:04,379 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 19:56:04,379 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(2446): Flush status journal for b1a5a8585f5b1218e278f2f033438f28: 2023-03-08 19:56:04,380 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/a0d713d9a8a741379b3b30e8e0438c67 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMinorCompactionWithMonthlyPolicy__rj22chn5irh4gpu0q2mmsjammch671lq9gl03114677jg2dgc81lnlng8potae1q/family1/a0d713d9a8a741379b3b30e8e0438c67 2023-03-08 19:56:04,382 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMinorCompactionWithMonthlyPolicy__rj22chn5irh4gpu0q2mmsjammch671lq9gl03114677jg2dgc81lnlng8potae1q/family1/a0d713d9a8a741379b3b30e8e0438c67 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/c3e7414d8a6443f49be8a38309b95372_SeqId_47_ 2023-03-08 19:56:04,383 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(849): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMinorCompactionWithMonthlyPolicy__rj22chn5irh4gpu0q2mmsjammch671lq9gl03114677jg2dgc81lnlng8potae1q/family1/a0d713d9a8a741379b3b30e8e0438c67 into b1a5a8585f5b1218e278f2f033438f28/family1 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/c3e7414d8a6443f49be8a38309b95372_SeqId_47_ - updating store file list. 2023-03-08 19:56:04,385 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(882): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/c3e7414d8a6443f49be8a38309b95372_SeqId_47_ into b1a5a8585f5b1218e278f2f033438f28/family1 2023-03-08 19:56:04,386 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(855): Successfully loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMinorCompactionWithMonthlyPolicy__rj22chn5irh4gpu0q2mmsjammch671lq9gl03114677jg2dgc81lnlng8potae1q/family1/a0d713d9a8a741379b3b30e8e0438c67 into b1a5a8585f5b1218e278f2f033438f28/family1 (new location: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/c3e7414d8a6443f49be8a38309b95372_SeqId_47_) 2023-03-08 19:56:04,386 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMinorCompactionWithMonthlyPolicy__rj22chn5irh4gpu0q2mmsjammch671lq9gl03114677jg2dgc81lnlng8potae1q/family1/a0d713d9a8a741379b3b30e8e0438c67 2023-03-08 19:56:04,386 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-08 19:56:04,387 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-08 19:56:04,387 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(1912): b1a5a8585f5b1218e278f2f033438f28/family1 is initiating minor compaction (all files) 2023-03-08 19:56:04,387 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@1f95c09a; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 19:56:04,387 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of b1a5a8585f5b1218e278f2f033438f28/family1 in testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. 2023-03-08 19:56:04,387 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/b4560772cc64412d807e338712b29416, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/e8acbef22de349189405241d63e6a1f0, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/c3e7414d8a6443f49be8a38309b95372_SeqId_47_] into tmpdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/.tmp, totalSize=178.1 K 2023-03-08 19:56:04,387 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting b4560772cc64412d807e338712b29416, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878843488 2023-03-08 19:56:04,388 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting e8acbef22de349189405241d63e6a1f0, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621243488 2023-03-08 19:56:04,388 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] compactions.Compactor(207): Compacting c3e7414d8a6443f49be8a38309b95372_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965243488 2023-03-08 19:56:04,389 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 19:56:04,389 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:56:04,390 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201db5bde649d484533bc1326b4f5e1f43c to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201db5bde649d484533bc1326b4f5e1f43c 2023-03-08 19:56:04,392 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205092b41f5a63340b4b41362d7cdfbe42f to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205092b41f5a63340b4b41362d7cdfbe42f 2023-03-08 19:56:04,393 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512285d2435c2383147f69c5245480074a167 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512285d2435c2383147f69c5245480074a167 2023-03-08 19:56:04,395 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015123149d94bab47fe487ebfd2353d976412e4 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015123149d94bab47fe487ebfd2353d976412e4 2023-03-08 19:56:04,395 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-03-08 19:56:04,395 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-03-08 19:56:04,395 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 19:56:04,395 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 19:56:04,403 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] throttle.PressureAwareThroughputController(145): b1a5a8585f5b1218e278f2f033438f28#family1#compaction#167 average throughput is 19.45 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 19:56:04,404 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/b1ea3450cbe64b5f825bcf72b5bf6bee, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/ab62f17ff0404c589c0cf688ef2455ff, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/6dcb2189f2a1443b9348cbd52bcad0be, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/6a4cbc7fab364ff991d5138d20b7633d, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/cc5e9db77f1348b19f4454e070bd8e7d, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/2901885445234e4aa3ffd1a5a8c72fe4, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/5eb45d065a294ab4b8f2a4f0a614c905, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/e75d6b26392c448c97ca37026fe91afe, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/7e3a5997519444b5925d1bc746cd1175, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/e374ff65a6c347f8a656b0b552677c63_SeqId_44_] to archive 2023-03-08 19:56:04,405 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:56:04,407 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/b1ea3450cbe64b5f825bcf72b5bf6bee to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/b1ea3450cbe64b5f825bcf72b5bf6bee 2023-03-08 19:56:04,410 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/ab62f17ff0404c589c0cf688ef2455ff to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/ab62f17ff0404c589c0cf688ef2455ff 2023-03-08 19:56:04,412 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/6dcb2189f2a1443b9348cbd52bcad0be to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/6dcb2189f2a1443b9348cbd52bcad0be 2023-03-08 19:56:04,413 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/6a4cbc7fab364ff991d5138d20b7633d to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/6a4cbc7fab364ff991d5138d20b7633d 2023-03-08 19:56:04,414 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/cc5e9db77f1348b19f4454e070bd8e7d to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/cc5e9db77f1348b19f4454e070bd8e7d 2023-03-08 19:56:04,415 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/2901885445234e4aa3ffd1a5a8c72fe4 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/2901885445234e4aa3ffd1a5a8c72fe4 2023-03-08 19:56:04,416 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/5eb45d065a294ab4b8f2a4f0a614c905 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/5eb45d065a294ab4b8f2a4f0a614c905 2023-03-08 19:56:04,417 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/e75d6b26392c448c97ca37026fe91afe to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/e75d6b26392c448c97ca37026fe91afe 2023-03-08 19:56:04,418 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/7e3a5997519444b5925d1bc746cd1175 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/7e3a5997519444b5925d1bc746cd1175 2023-03-08 19:56:04,419 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/e374ff65a6c347f8a656b0b552677c63_SeqId_44_ to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/e374ff65a6c347f8a656b0b552677c63_SeqId_44_ 2023-03-08 19:56:04,829 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/.tmp/family1/2f3ea986c27c4b92a1d73dc365f209aa as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/2f3ea986c27c4b92a1d73dc365f209aa 2023-03-08 19:56:04,834 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in b1a5a8585f5b1218e278f2f033438f28/family1 of b1a5a8585f5b1218e278f2f033438f28 into 2f3ea986c27c4b92a1d73dc365f209aa(size=167.4 K), total size for store is 167.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 19:56:04,834 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for b1a5a8585f5b1218e278f2f033438f28: 2023-03-08 19:56:04,835 INFO [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1678305364387; duration=0sec 2023-03-08 19:56:04,835 DEBUG [RS:0;jenkins-hbase4:33367-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-08 19:56:04,897 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/b4560772cc64412d807e338712b29416, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/e8acbef22de349189405241d63e6a1f0, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/c3e7414d8a6443f49be8a38309b95372_SeqId_47_] to archive 2023-03-08 19:56:04,898 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:56:04,900 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/b4560772cc64412d807e338712b29416 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/b4560772cc64412d807e338712b29416 2023-03-08 19:56:04,901 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/e8acbef22de349189405241d63e6a1f0 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/e8acbef22de349189405241d63e6a1f0 2023-03-08 19:56:04,902 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/c3e7414d8a6443f49be8a38309b95372_SeqId_47_ to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/c3e7414d8a6443f49be8a38309b95372_SeqId_47_ 2023-03-08 19:56:05,241 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130420bc542ab034888854f8affb0594ad5 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130420bc542ab034888854f8affb0594ad5 2023-03-08 19:56:05,242 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 19:56:05,271 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/3c4d83e3bccd40909a275bdf2b565cbf first=Optional[r00] last=Optional[r8] 2023-03-08 19:56:05,272 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28., hostname=jenkins-hbase4.apache.org,33367,1678305218680, seqNum=2 for row with hfile group [{family1,hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/3c4d83e3bccd40909a275bdf2b565cbf}] 2023-03-08 19:56:05,274 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(763): Validating hfile at hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/3c4d83e3bccd40909a275bdf2b565cbf for inclusion in b1a5a8585f5b1218e278f2f033438f28/family1 2023-03-08 19:56:05,276 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-03-08 19:56:05,276 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(777): Region bounds: first= last= 2023-03-08 19:56:05,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(2446): Flush status journal for b1a5a8585f5b1218e278f2f033438f28: 2023-03-08 19:56:05,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/3c4d83e3bccd40909a275bdf2b565cbf to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMinorCompactionWithMonthlyPolicy__q644robct00v0npovvo2558vqpknlbug6i0psqjdi1o3euu49oaj6aocfh4lhocr/family1/3c4d83e3bccd40909a275bdf2b565cbf 2023-03-08 19:56:05,279 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMinorCompactionWithMonthlyPolicy__q644robct00v0npovvo2558vqpknlbug6i0psqjdi1o3euu49oaj6aocfh4lhocr/family1/3c4d83e3bccd40909a275bdf2b565cbf as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/7aa109e217184fcbba8e04cc00711b31_SeqId_50_ 2023-03-08 19:56:05,280 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(849): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMinorCompactionWithMonthlyPolicy__q644robct00v0npovvo2558vqpknlbug6i0psqjdi1o3euu49oaj6aocfh4lhocr/family1/3c4d83e3bccd40909a275bdf2b565cbf into b1a5a8585f5b1218e278f2f033438f28/family1 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/7aa109e217184fcbba8e04cc00711b31_SeqId_50_ - updating store file list. 2023-03-08 19:56:05,284 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(882): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/7aa109e217184fcbba8e04cc00711b31_SeqId_50_ into b1a5a8585f5b1218e278f2f033438f28/family1 2023-03-08 19:56:05,284 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(855): Successfully loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMinorCompactionWithMonthlyPolicy__q644robct00v0npovvo2558vqpknlbug6i0psqjdi1o3euu49oaj6aocfh4lhocr/family1/3c4d83e3bccd40909a275bdf2b565cbf into b1a5a8585f5b1218e278f2f033438f28/family1 (new location: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/family1/7aa109e217184fcbba8e04cc00711b31_SeqId_50_) 2023-03-08 19:56:05,284 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMinorCompactionWithMonthlyPolicy__q644robct00v0npovvo2558vqpknlbug6i0psqjdi1o3euu49oaj6aocfh4lhocr/family1/3c4d83e3bccd40909a275bdf2b565cbf 2023-03-08 19:56:05,286 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 19:56:05,287 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:56:05,288 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120e3597676df8946e7bc7c07bdcbe0a24a to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120e3597676df8946e7bc7c07bdcbe0a24a 2023-03-08 19:56:05,289 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128ee847ffd5df7493e8831556bc39f3838 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128ee847ffd5df7493e8831556bc39f3838 2023-03-08 19:56:05,290 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511302192fc08b4b64ea2905586cc213b98bd to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511302192fc08b4b64ea2905586cc213b98bd 2023-03-08 19:56:05,290 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-03-08 19:56:05,290 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-08 19:56:05,290 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 19:56:05,290 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 19:56:05,291 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-08 19:56:05,291 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-08 19:56:05,291 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 19:56:05,292 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=115, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-08 19:56:05,293 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=115, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 1.8940 sec 2023-03-08 19:56:05,483 INFO [Listener at localhost/34049] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=257 (was 248) - Thread LEAK? -, OpenFileDescriptor=654 (was 625) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=187 (was 137) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=8185 (was 8198) 2023-03-08 19:56:05,495 INFO [Listener at localhost/34049] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=257, OpenFileDescriptor=654, MaxFileDescriptor=60000, SystemLoadAverage=187, ProcessCount=170, AvailableMemoryMB=8184 2023-03-08 19:56:05,514 DEBUG [Listener at localhost/34049] crypto.Encryption(565): Installed org.apache.hadoop.hbase.io.crypto.KeyProviderForTesting into key provider cache 2023-03-08 19:56:05,568 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xBDx\xE3<\x98\xBF\xFAP\xED\xDC\xC5\xDCr\x1D\xF4\x11"\x10pG-\xB6\x80~\xD7\xD6\xCCw\xCEC\x03\x15\xE8\xB3*\x10\xE1\xB9\x82+9\x12$eX3\xC9n\x07-\xE2\x002\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-08 19:56:05,569 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] procedure2.ProcedureExecutor(1029): Stored pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionFromAdmin 2023-03-08 19:56:05,570 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_PRE_OPERATION 2023-03-08 19:56:05,570 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionFromAdmin" procId is: 116 2023-03-08 19:56:05,571 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-08 19:56:05,571 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-08 19:56:05,574 DEBUG [HFileArchiver-20] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811 2023-03-08 19:56:05,574 DEBUG [HFileArchiver-21] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7 2023-03-08 19:56:05,574 DEBUG [HFileArchiver-22] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9 2023-03-08 19:56:05,574 DEBUG [HFileArchiver-20] backup.HFileArchiver(153): Directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811 empty. 2023-03-08 19:56:05,574 DEBUG [HFileArchiver-22] backup.HFileArchiver(153): Directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9 empty. 2023-03-08 19:56:05,574 DEBUG [HFileArchiver-21] backup.HFileArchiver(153): Directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7 empty. 2023-03-08 19:56:05,575 DEBUG [HFileArchiver-20] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811 2023-03-08 19:56:05,575 DEBUG [HFileArchiver-22] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9 2023-03-08 19:56:05,575 DEBUG [HFileArchiver-21] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7 2023-03-08 19:56:05,575 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMajorCompactionFromAdmin regions 2023-03-08 19:56:05,672 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-08 19:56:05,873 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-08 19:56:05,982 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp/data/default/testMajorCompactionFromAdmin/.tabledesc/.tableinfo.0000000001 2023-03-08 19:56:05,984 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(7675): creating {ENCODED => 0abcf5c9f041342bc11db8ee9ff848c9, NAME => 'testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xBDx\xE3<\x98\xBF\xFAP\xED\xDC\xC5\xDCr\x1D\xF4\x11"\x10pG-\xB6\x80~\xD7\xD6\xCCw\xCEC\x03\x15\xE8\xB3*\x10\xE1\xB9\x82+9\x12$eX3\xC9n\x07-\xE2\x002\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp 2023-03-08 19:56:05,984 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(7675): creating {ENCODED => 78278c90009823fdb2b2cd7994a08811, NAME => 'testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xBDx\xE3<\x98\xBF\xFAP\xED\xDC\xC5\xDCr\x1D\xF4\x11"\x10pG-\xB6\x80~\xD7\xD6\xCCw\xCEC\x03\x15\xE8\xB3*\x10\xE1\xB9\x82+9\x12$eX3\xC9n\x07-\xE2\x002\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp 2023-03-08 19:56:05,984 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(7675): creating {ENCODED => 7becb875ae4303bd834b15dde14376f7, NAME => 'testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xBDx\xE3<\x98\xBF\xFAP\xED\xDC\xC5\xDCr\x1D\xF4\x11"\x10pG-\xB6\x80~\xD7\xD6\xCCw\xCEC\x03\x15\xE8\xB3*\x10\xE1\xB9\x82+9\x12$eX3\xC9n\x07-\xE2\x002\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/.tmp 2023-03-08 19:56:06,174 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-08 19:56:06,389 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:56:06,389 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:56:06,389 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1604): Closing 0abcf5c9f041342bc11db8ee9ff848c9, disabling compactions & flushes 2023-03-08 19:56:06,390 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1604): Closing 78278c90009823fdb2b2cd7994a08811, disabling compactions & flushes 2023-03-08 19:56:06,390 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9. 2023-03-08 19:56:06,390 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811. 2023-03-08 19:56:06,390 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811. 2023-03-08 19:56:06,390 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:56:06,390 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9. 2023-03-08 19:56:06,390 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1604): Closing 7becb875ae4303bd834b15dde14376f7, disabling compactions & flushes 2023-03-08 19:56:06,390 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811. after waiting 0 ms 2023-03-08 19:56:06,390 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7. 2023-03-08 19:56:06,390 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7. 2023-03-08 19:56:06,390 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811. 2023-03-08 19:56:06,390 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7. after waiting 0 ms 2023-03-08 19:56:06,390 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811. 2023-03-08 19:56:06,390 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7. 2023-03-08 19:56:06,390 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9. after waiting 0 ms 2023-03-08 19:56:06,390 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7. 2023-03-08 19:56:06,390 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1558): Region close journal for 78278c90009823fdb2b2cd7994a08811: 2023-03-08 19:56:06,390 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1558): Region close journal for 7becb875ae4303bd834b15dde14376f7: 2023-03-08 19:56:06,390 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9. 2023-03-08 19:56:06,390 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9. 2023-03-08 19:56:06,390 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1558): Region close journal for 0abcf5c9f041342bc11db8ee9ff848c9: 2023-03-08 19:56:06,392 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ADD_TO_META 2023-03-08 19:56:06,393 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678305366393"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678305366393"}]},"ts":"1678305366393"} 2023-03-08 19:56:06,393 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1678305366393"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678305366393"}]},"ts":"1678305366393"} 2023-03-08 19:56:06,393 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678305366393"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678305366393"}]},"ts":"1678305366393"} 2023-03-08 19:56:06,395 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-08 19:56:06,395 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-08 19:56:06,395 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678305366395"}]},"ts":"1678305366395"} 2023-03-08 19:56:06,396 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLING in hbase:meta 2023-03-08 19:56:06,400 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=78278c90009823fdb2b2cd7994a08811, ASSIGN}, {pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=7becb875ae4303bd834b15dde14376f7, ASSIGN}, {pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=0abcf5c9f041342bc11db8ee9ff848c9, ASSIGN}] 2023-03-08 19:56:06,402 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=0abcf5c9f041342bc11db8ee9ff848c9, ASSIGN 2023-03-08 19:56:06,402 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=7becb875ae4303bd834b15dde14376f7, ASSIGN 2023-03-08 19:56:06,402 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=78278c90009823fdb2b2cd7994a08811, ASSIGN 2023-03-08 19:56:06,402 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=0abcf5c9f041342bc11db8ee9ff848c9, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33367,1678305218680; forceNewPlan=false, retain=false 2023-03-08 19:56:06,402 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=7becb875ae4303bd834b15dde14376f7, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33367,1678305218680; forceNewPlan=false, retain=false 2023-03-08 19:56:06,402 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=78278c90009823fdb2b2cd7994a08811, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33367,1678305218680; forceNewPlan=false, retain=false 2023-03-08 19:56:06,554 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=78278c90009823fdb2b2cd7994a08811, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:06,554 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=0abcf5c9f041342bc11db8ee9ff848c9, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:06,554 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=7becb875ae4303bd834b15dde14376f7, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:06,554 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678305366554"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678305366554"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678305366554"}]},"ts":"1678305366554"} 2023-03-08 19:56:06,554 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1678305366554"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678305366554"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678305366554"}]},"ts":"1678305366554"} 2023-03-08 19:56:06,554 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678305366554"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678305366554"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678305366554"}]},"ts":"1678305366554"} 2023-03-08 19:56:06,556 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=117, state=RUNNABLE; OpenRegionProcedure 78278c90009823fdb2b2cd7994a08811, server=jenkins-hbase4.apache.org,33367,1678305218680}] 2023-03-08 19:56:06,556 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=121, ppid=118, state=RUNNABLE; OpenRegionProcedure 7becb875ae4303bd834b15dde14376f7, server=jenkins-hbase4.apache.org,33367,1678305218680}] 2023-03-08 19:56:06,557 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=119, state=RUNNABLE; OpenRegionProcedure 0abcf5c9f041342bc11db8ee9ff848c9, server=jenkins-hbase4.apache.org,33367,1678305218680}] 2023-03-08 19:56:06,675 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-08 19:56:06,709 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811. 2023-03-08 19:56:06,709 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 78278c90009823fdb2b2cd7994a08811, NAME => 'testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811.', STARTKEY => '', ENDKEY => '1'} 2023-03-08 19:56:06,710 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 78278c90009823fdb2b2cd7994a08811 2023-03-08 19:56:06,710 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:56:06,710 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 78278c90009823fdb2b2cd7994a08811 2023-03-08 19:56:06,710 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 78278c90009823fdb2b2cd7994a08811 2023-03-08 19:56:06,731 INFO [StoreOpener-78278c90009823fdb2b2cd7994a08811-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 78278c90009823fdb2b2cd7994a08811 2023-03-08 19:56:06,733 DEBUG [StoreOpener-78278c90009823fdb2b2cd7994a08811-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family1 2023-03-08 19:56:06,733 DEBUG [StoreOpener-78278c90009823fdb2b2cd7994a08811-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family1 2023-03-08 19:56:06,733 INFO [StoreOpener-78278c90009823fdb2b2cd7994a08811-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 78278c90009823fdb2b2cd7994a08811 columnFamilyName family1 2023-03-08 19:56:06,734 INFO [StoreOpener-78278c90009823fdb2b2cd7994a08811-1] regionserver.HStore(310): Store=78278c90009823fdb2b2cd7994a08811/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:56:06,735 INFO [StoreOpener-78278c90009823fdb2b2cd7994a08811-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 78278c90009823fdb2b2cd7994a08811 2023-03-08 19:56:06,736 DEBUG [StoreOpener-78278c90009823fdb2b2cd7994a08811-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family2 2023-03-08 19:56:06,736 DEBUG [StoreOpener-78278c90009823fdb2b2cd7994a08811-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family2 2023-03-08 19:56:06,736 INFO [StoreOpener-78278c90009823fdb2b2cd7994a08811-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 78278c90009823fdb2b2cd7994a08811 columnFamilyName family2 2023-03-08 19:56:06,737 INFO [StoreOpener-78278c90009823fdb2b2cd7994a08811-1] regionserver.HStore(310): Store=78278c90009823fdb2b2cd7994a08811/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:56:06,738 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811 2023-03-08 19:56:06,738 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811 2023-03-08 19:56:06,739 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 19:56:06,740 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 78278c90009823fdb2b2cd7994a08811 2023-03-08 19:56:06,742 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 19:56:06,742 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 78278c90009823fdb2b2cd7994a08811; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63865026, jitterRate=-0.048336952924728394}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 19:56:06,742 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 78278c90009823fdb2b2cd7994a08811: 2023-03-08 19:56:06,743 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811., pid=120, masterSystemTime=1678305366707 2023-03-08 19:56:06,744 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811. 2023-03-08 19:56:06,744 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811. 2023-03-08 19:56:06,744 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9. 2023-03-08 19:56:06,744 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 0abcf5c9f041342bc11db8ee9ff848c9, NAME => 'testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9.', STARTKEY => '2', ENDKEY => ''} 2023-03-08 19:56:06,745 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=78278c90009823fdb2b2cd7994a08811, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:06,745 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 0abcf5c9f041342bc11db8ee9ff848c9 2023-03-08 19:56:06,745 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678305366744"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678305366744"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678305366744"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678305366744"}]},"ts":"1678305366744"} 2023-03-08 19:56:06,745 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:56:06,745 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 0abcf5c9f041342bc11db8ee9ff848c9 2023-03-08 19:56:06,745 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 0abcf5c9f041342bc11db8ee9ff848c9 2023-03-08 19:56:06,747 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=117 2023-03-08 19:56:06,747 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=117, state=SUCCESS; OpenRegionProcedure 78278c90009823fdb2b2cd7994a08811, server=jenkins-hbase4.apache.org,33367,1678305218680 in 190 msec 2023-03-08 19:56:06,748 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=78278c90009823fdb2b2cd7994a08811, ASSIGN in 347 msec 2023-03-08 19:56:06,765 INFO [StoreOpener-0abcf5c9f041342bc11db8ee9ff848c9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 0abcf5c9f041342bc11db8ee9ff848c9 2023-03-08 19:56:06,766 DEBUG [StoreOpener-0abcf5c9f041342bc11db8ee9ff848c9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family1 2023-03-08 19:56:06,767 DEBUG [StoreOpener-0abcf5c9f041342bc11db8ee9ff848c9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family1 2023-03-08 19:56:06,767 INFO [StoreOpener-0abcf5c9f041342bc11db8ee9ff848c9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0abcf5c9f041342bc11db8ee9ff848c9 columnFamilyName family1 2023-03-08 19:56:06,767 INFO [StoreOpener-0abcf5c9f041342bc11db8ee9ff848c9-1] regionserver.HStore(310): Store=0abcf5c9f041342bc11db8ee9ff848c9/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:56:06,768 INFO [StoreOpener-0abcf5c9f041342bc11db8ee9ff848c9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 0abcf5c9f041342bc11db8ee9ff848c9 2023-03-08 19:56:06,768 DEBUG [StoreOpener-0abcf5c9f041342bc11db8ee9ff848c9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family2 2023-03-08 19:56:06,769 DEBUG [StoreOpener-0abcf5c9f041342bc11db8ee9ff848c9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family2 2023-03-08 19:56:06,769 INFO [StoreOpener-0abcf5c9f041342bc11db8ee9ff848c9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0abcf5c9f041342bc11db8ee9ff848c9 columnFamilyName family2 2023-03-08 19:56:06,769 INFO [StoreOpener-0abcf5c9f041342bc11db8ee9ff848c9-1] regionserver.HStore(310): Store=0abcf5c9f041342bc11db8ee9ff848c9/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:56:06,770 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9 2023-03-08 19:56:06,770 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9 2023-03-08 19:56:06,771 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 19:56:06,772 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 0abcf5c9f041342bc11db8ee9ff848c9 2023-03-08 19:56:06,773 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 19:56:06,774 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 0abcf5c9f041342bc11db8ee9ff848c9; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61354699, jitterRate=-0.0857437402009964}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 19:56:06,774 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 0abcf5c9f041342bc11db8ee9ff848c9: 2023-03-08 19:56:06,774 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9., pid=122, masterSystemTime=1678305366707 2023-03-08 19:56:06,775 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9. 2023-03-08 19:56:06,775 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9. 2023-03-08 19:56:06,775 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7. 2023-03-08 19:56:06,775 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 7becb875ae4303bd834b15dde14376f7, NAME => 'testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7.', STARTKEY => '1', ENDKEY => '2'} 2023-03-08 19:56:06,776 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 7becb875ae4303bd834b15dde14376f7 2023-03-08 19:56:06,776 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=0abcf5c9f041342bc11db8ee9ff848c9, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:06,776 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-08 19:56:06,776 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 7becb875ae4303bd834b15dde14376f7 2023-03-08 19:56:06,776 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 7becb875ae4303bd834b15dde14376f7 2023-03-08 19:56:06,776 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678305366776"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678305366776"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678305366776"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678305366776"}]},"ts":"1678305366776"} 2023-03-08 19:56:06,778 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=119 2023-03-08 19:56:06,778 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=119, state=SUCCESS; OpenRegionProcedure 0abcf5c9f041342bc11db8ee9ff848c9, server=jenkins-hbase4.apache.org,33367,1678305218680 in 220 msec 2023-03-08 19:56:06,779 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=119, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=0abcf5c9f041342bc11db8ee9ff848c9, ASSIGN in 378 msec 2023-03-08 19:56:06,796 INFO [StoreOpener-7becb875ae4303bd834b15dde14376f7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 7becb875ae4303bd834b15dde14376f7 2023-03-08 19:56:06,797 DEBUG [StoreOpener-7becb875ae4303bd834b15dde14376f7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family1 2023-03-08 19:56:06,797 DEBUG [StoreOpener-7becb875ae4303bd834b15dde14376f7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family1 2023-03-08 19:56:06,798 INFO [StoreOpener-7becb875ae4303bd834b15dde14376f7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7becb875ae4303bd834b15dde14376f7 columnFamilyName family1 2023-03-08 19:56:06,798 INFO [StoreOpener-7becb875ae4303bd834b15dde14376f7-1] regionserver.HStore(310): Store=7becb875ae4303bd834b15dde14376f7/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:56:06,799 INFO [StoreOpener-7becb875ae4303bd834b15dde14376f7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 7becb875ae4303bd834b15dde14376f7 2023-03-08 19:56:06,799 DEBUG [StoreOpener-7becb875ae4303bd834b15dde14376f7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family2 2023-03-08 19:56:06,799 DEBUG [StoreOpener-7becb875ae4303bd834b15dde14376f7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family2 2023-03-08 19:56:06,800 INFO [StoreOpener-7becb875ae4303bd834b15dde14376f7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7becb875ae4303bd834b15dde14376f7 columnFamilyName family2 2023-03-08 19:56:06,800 INFO [StoreOpener-7becb875ae4303bd834b15dde14376f7-1] regionserver.HStore(310): Store=7becb875ae4303bd834b15dde14376f7/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-08 19:56:06,801 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7 2023-03-08 19:56:06,801 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7 2023-03-08 19:56:06,802 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-08 19:56:06,803 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 7becb875ae4303bd834b15dde14376f7 2023-03-08 19:56:06,804 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-08 19:56:06,805 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 7becb875ae4303bd834b15dde14376f7; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73455161, jitterRate=0.0945671945810318}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-08 19:56:06,805 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 7becb875ae4303bd834b15dde14376f7: 2023-03-08 19:56:06,805 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7., pid=121, masterSystemTime=1678305366707 2023-03-08 19:56:06,807 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7. 2023-03-08 19:56:06,807 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7. 2023-03-08 19:56:06,807 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=7becb875ae4303bd834b15dde14376f7, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:06,807 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1678305366807"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678305366807"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678305366807"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678305366807"}]},"ts":"1678305366807"} 2023-03-08 19:56:06,810 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=121, resume processing ppid=118 2023-03-08 19:56:06,810 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=118, state=SUCCESS; OpenRegionProcedure 7becb875ae4303bd834b15dde14376f7, server=jenkins-hbase4.apache.org,33367,1678305218680 in 253 msec 2023-03-08 19:56:06,811 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=116 2023-03-08 19:56:06,811 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=7becb875ae4303bd834b15dde14376f7, ASSIGN in 410 msec 2023-03-08 19:56:06,812 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-08 19:56:06,812 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678305366812"}]},"ts":"1678305366812"} 2023-03-08 19:56:06,813 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLED in hbase:meta 2023-03-08 19:56:06,815 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_POST_OPERATION 2023-03-08 19:56:06,817 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=116, state=SUCCESS; CreateTableProcedure table=testMajorCompactionFromAdmin in 1.2480 sec 2023-03-08 19:56:07,676 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-08 19:56:07,676 INFO [Listener at localhost/34049] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionFromAdmin, procId: 116 completed 2023-03-08 19:56:07,682 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 19:56:07,683 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 19:56:07,683 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 19:56:07,685 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:56:07,686 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-08 19:56:07,686 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-08 19:56:07,686 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:56:07,686 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-08 19:56:07,686 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-08 19:56:07,687 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:07,687 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 19:56:07,688 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:56:07,689 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:56:07,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:56:07,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 19:56:07,689 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:07,689 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:07,689 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:07,689 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:56:07,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:56:07,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 19:56:07,690 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-08 19:56:07,690 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-08 19:56:07,690 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-08 19:56:07,690 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:56:07,690 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-08 19:56:07,690 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9. 2023-03-08 19:56:07,690 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9. started... 2023-03-08 19:56:07,690 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811. 2023-03-08 19:56:07,690 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811. started... 2023-03-08 19:56:07,697 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 19:56:07,704 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0abcf5c9f041342bc11db8ee9ff848c9 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-08 19:56:07,704 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:56:07,704 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7. 2023-03-08 19:56:07,704 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 78278c90009823fdb2b2cd7994a08811 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-08 19:56:07,705 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7. started... 2023-03-08 19:56:07,705 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 7becb875ae4303bd834b15dde14376f7 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-08 19:56:07,756 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303085c3a777593b347e89dcc23570f0be7c8 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303085c3a777593b347e89dcc23570f0be7c8 2023-03-08 19:56:07,756 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308e348a7514c2f4365ab01865dd88c95c1 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230308e348a7514c2f4365ab01865dd88c95c1 2023-03-08 19:56:07,760 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303085cabfd51d72e4e5fb2676b8238f51919 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303085cabfd51d72e4e5fb2676b8238f51919 2023-03-08 19:56:07,770 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/.tmp/family1/b4a72f07e25c4cd1b4b80a9c7684653d 2023-03-08 19:56:07,777 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/.tmp/family1/3df95a65498e45a4841c4a31b6a10750 2023-03-08 19:56:08,176 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/.tmp/family1/57923c96e6164f0a9ad738427b68febd 2023-03-08 19:56:08,209 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308358d0834605f48198b90985f2184fa05 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230308358d0834605f48198b90985f2184fa05 2023-03-08 19:56:08,215 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303087262ba7df13847a9937dd006da9391c2 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b202303087262ba7df13847a9937dd006da9391c2 2023-03-08 19:56:08,608 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303089a59fa92d02b4e77807e0787f804ac63 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202303089a59fa92d02b4e77807e0787f804ac63 2023-03-08 19:56:08,614 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/.tmp/family2/8b7079ad98e640428777ff32b0dbab45 2023-03-08 19:56:08,617 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/.tmp/family1/b4a72f07e25c4cd1b4b80a9c7684653d as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family1/b4a72f07e25c4cd1b4b80a9c7684653d 2023-03-08 19:56:08,619 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/.tmp/family2/107493347ac944efa7a27877fd5ec384 2023-03-08 19:56:08,622 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/.tmp/family1/3df95a65498e45a4841c4a31b6a10750 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family1/3df95a65498e45a4841c4a31b6a10750 2023-03-08 19:56:08,640 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family1/b4a72f07e25c4cd1b4b80a9c7684653d, entries=4, sequenceid=5, filesize=5.5 K 2023-03-08 19:56:08,641 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/.tmp/family2/8b7079ad98e640428777ff32b0dbab45 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family2/8b7079ad98e640428777ff32b0dbab45 2023-03-08 19:56:08,644 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family1/3df95a65498e45a4841c4a31b6a10750, entries=4, sequenceid=5, filesize=5.5 K 2023-03-08 19:56:08,645 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/.tmp/family2/107493347ac944efa7a27877fd5ec384 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family2/107493347ac944efa7a27877fd5ec384 2023-03-08 19:56:08,645 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family2/8b7079ad98e640428777ff32b0dbab45, entries=2, sequenceid=5, filesize=5.1 K 2023-03-08 19:56:08,646 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 78278c90009823fdb2b2cd7994a08811 in 942ms, sequenceid=5, compaction requested=false 2023-03-08 19:56:08,646 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionFromAdmin' 2023-03-08 19:56:08,646 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 78278c90009823fdb2b2cd7994a08811: 2023-03-08 19:56:08,646 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811. 2023-03-08 19:56:08,646 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 19:56:08,648 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family2/107493347ac944efa7a27877fd5ec384, entries=2, sequenceid=5, filesize=5.1 K 2023-03-08 19:56:08,649 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 7becb875ae4303bd834b15dde14376f7 in 944ms, sequenceid=5, compaction requested=false 2023-03-08 19:56:08,650 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 7becb875ae4303bd834b15dde14376f7: 2023-03-08 19:56:08,650 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7. 2023-03-08 19:56:08,650 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 19:56:09,013 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/.tmp/family2/f47a02760b4f4175bec133816380b0c1 2023-03-08 19:56:09,016 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/.tmp/family1/57923c96e6164f0a9ad738427b68febd as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family1/57923c96e6164f0a9ad738427b68febd 2023-03-08 19:56:09,039 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family1/57923c96e6164f0a9ad738427b68febd, entries=4, sequenceid=5, filesize=5.5 K 2023-03-08 19:56:09,040 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/.tmp/family2/f47a02760b4f4175bec133816380b0c1 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family2/f47a02760b4f4175bec133816380b0c1 2023-03-08 19:56:09,043 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family2/f47a02760b4f4175bec133816380b0c1, entries=2, sequenceid=5, filesize=5.1 K 2023-03-08 19:56:09,044 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 0abcf5c9f041342bc11db8ee9ff848c9 in 1340ms, sequenceid=5, compaction requested=false 2023-03-08 19:56:09,044 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0abcf5c9f041342bc11db8ee9ff848c9: 2023-03-08 19:56:09,044 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9. 2023-03-08 19:56:09,044 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 19:56:09,044 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 19:56:09,044 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:09,044 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-08 19:56:09,044 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-08 19:56:09,049 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:09,049 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 19:56:09,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:09,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:56:09,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:56:09,050 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 19:56:09,050 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:56:09,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:56:09,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:56:09,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 19:56:09,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:09,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:56:09,051 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-08 19:56:09,051 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@78572b32[Count = 0] remaining members to acquire global barrier 2023-03-08 19:56:09,051 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-08 19:56:09,051 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 19:56:09,052 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 19:56:09,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 19:56:09,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 19:56:09,052 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-08 19:56:09,052 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-08 19:56:09,052 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:56:09,052 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:09,052 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:56:09,053 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:09,053 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-08 19:56:09,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:09,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:56:09,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:56:09,053 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:56:09,054 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-08 19:56:09,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:56:09,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:56:09,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 19:56:09,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:09,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:56:09,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 19:56:09,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:09,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:56:09,055 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-08 19:56:09,055 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-08 19:56:09,055 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:56:09,055 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:56:09,055 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-08 19:56:09,055 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:56:09,058 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:09,058 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:09,058 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:56:09,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:09,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:56:09,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:56:09,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:09,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:09,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:56:09,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:56:09,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:56:09,058 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:09,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 19:56:09,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:09,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:56:09,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 19:56:09,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:09,059 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:09,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:56:09,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 19:56:09,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:09,061 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:09,061 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:56:09,062 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 19:56:09,062 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:56:09,062 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-08 19:56:09,062 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:56:09,062 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:56:09,062 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 19:56:09,062 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:56:09,062 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:09,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:56:09,062 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 19:56:09,062 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 19:56:09,062 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-08 19:56:09,062 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:09,062 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:56:09,062 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:56:09,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:56:09,162 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:56:09,163 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:56:09,164 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 19:56:09,164 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 19:56:09,165 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 19:56:09,166 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:56:09,168 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-08 19:56:09,168 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-08 19:56:09,168 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-08 19:56:09,168 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:56:09,168 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-08 19:56:09,168 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-08 19:56:09,168 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:09,169 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 19:56:09,170 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:09,170 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:56:09,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:56:09,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:56:09,170 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:09,170 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:56:09,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 19:56:09,170 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:09,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:56:09,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 19:56:09,171 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-08 19:56:09,171 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-08 19:56:09,171 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-08 19:56:09,171 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-08 19:56:09,171 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:56:09,171 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-08 19:56:09,171 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9. 2023-03-08 19:56:09,171 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9. started... 2023-03-08 19:56:09,171 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 19:56:09,171 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7. 2023-03-08 19:56:09,171 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0abcf5c9f041342bc11db8ee9ff848c9 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-08 19:56:09,171 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7. started... 2023-03-08 19:56:09,171 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:56:09,171 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811. 2023-03-08 19:56:09,171 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811. started... 2023-03-08 19:56:09,171 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 7becb875ae4303bd834b15dde14376f7 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-08 19:56:09,172 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 78278c90009823fdb2b2cd7994a08811 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-08 19:56:09,216 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308e8b5eadb54ef4028bf8e0819d2e4e30c to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230308e8b5eadb54ef4028bf8e0819d2e4e30c 2023-03-08 19:56:09,613 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303088f40467618774e95b811c224c93dba3d to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303088f40467618774e95b811c224c93dba3d 2023-03-08 19:56:09,616 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308f79f141dd7c84a5dad3f498a47157cc6 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230308f79f141dd7c84a5dad3f498a47157cc6 2023-03-08 19:56:09,631 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/.tmp/family1/62830d7c5b024fcd8575068c5b33cabf 2023-03-08 19:56:10,027 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/.tmp/family1/0ecd44f886594522b75e5da77e30afa3 2023-03-08 19:56:10,030 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/.tmp/family1/b7d8bbfffb884af7a33b742a441b4543 2023-03-08 19:56:10,061 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308252dcca862554d259c6084edebbf4235 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230308252dcca862554d259c6084edebbf4235 2023-03-08 19:56:10,461 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308a26d0e4833f04db39ac53a409c7285cc to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230308a26d0e4833f04db39ac53a409c7285cc 2023-03-08 19:56:10,461 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023030864861c6fd5c64531853d93951e6e886b to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b2023030864861c6fd5c64531853d93951e6e886b 2023-03-08 19:56:10,465 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/.tmp/family2/0f6a9c3780cc49fd8eecb5bffdefd4a9 2023-03-08 19:56:10,468 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/.tmp/family1/62830d7c5b024fcd8575068c5b33cabf as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family1/62830d7c5b024fcd8575068c5b33cabf 2023-03-08 19:56:10,492 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family1/62830d7c5b024fcd8575068c5b33cabf, entries=4, sequenceid=9, filesize=5.5 K 2023-03-08 19:56:10,492 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/.tmp/family2/0f6a9c3780cc49fd8eecb5bffdefd4a9 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family2/0f6a9c3780cc49fd8eecb5bffdefd4a9 2023-03-08 19:56:10,495 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family2/0f6a9c3780cc49fd8eecb5bffdefd4a9, entries=2, sequenceid=9, filesize=5.1 K 2023-03-08 19:56:10,496 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 0abcf5c9f041342bc11db8ee9ff848c9 in 1325ms, sequenceid=9, compaction requested=false 2023-03-08 19:56:10,496 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0abcf5c9f041342bc11db8ee9ff848c9: 2023-03-08 19:56:10,496 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9. 2023-03-08 19:56:10,496 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 19:56:10,865 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/.tmp/family2/2018f85e8b2244cb84b0f7f433314ebc 2023-03-08 19:56:10,867 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/.tmp/family2/a9804969245048d995d1e495e20df9ce 2023-03-08 19:56:10,870 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/.tmp/family1/b7d8bbfffb884af7a33b742a441b4543 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family1/b7d8bbfffb884af7a33b742a441b4543 2023-03-08 19:56:10,870 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/.tmp/family1/0ecd44f886594522b75e5da77e30afa3 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family1/0ecd44f886594522b75e5da77e30afa3 2023-03-08 19:56:10,893 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family1/b7d8bbfffb884af7a33b742a441b4543, entries=4, sequenceid=9, filesize=5.5 K 2023-03-08 19:56:10,893 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family1/0ecd44f886594522b75e5da77e30afa3, entries=4, sequenceid=9, filesize=5.5 K 2023-03-08 19:56:10,894 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/.tmp/family2/2018f85e8b2244cb84b0f7f433314ebc as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family2/2018f85e8b2244cb84b0f7f433314ebc 2023-03-08 19:56:10,894 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/.tmp/family2/a9804969245048d995d1e495e20df9ce as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family2/a9804969245048d995d1e495e20df9ce 2023-03-08 19:56:10,897 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family2/2018f85e8b2244cb84b0f7f433314ebc, entries=2, sequenceid=9, filesize=5.1 K 2023-03-08 19:56:10,897 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family2/a9804969245048d995d1e495e20df9ce, entries=2, sequenceid=9, filesize=5.1 K 2023-03-08 19:56:10,898 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 78278c90009823fdb2b2cd7994a08811 in 1726ms, sequenceid=9, compaction requested=false 2023-03-08 19:56:10,898 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 78278c90009823fdb2b2cd7994a08811: 2023-03-08 19:56:10,898 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811. 2023-03-08 19:56:10,898 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 7becb875ae4303bd834b15dde14376f7 in 1727ms, sequenceid=9, compaction requested=false 2023-03-08 19:56:10,898 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 19:56:10,898 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 7becb875ae4303bd834b15dde14376f7: 2023-03-08 19:56:10,898 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7. 2023-03-08 19:56:10,898 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 19:56:10,898 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 19:56:10,898 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:10,898 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-08 19:56:10,898 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-08 19:56:10,899 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:10,899 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 19:56:10,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:10,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:56:10,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:56:10,900 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 19:56:10,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:56:10,900 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:56:10,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:56:10,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 19:56:10,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:10,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:56:10,902 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-08 19:56:10,902 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5484c9c9[Count = 0] remaining members to acquire global barrier 2023-03-08 19:56:10,902 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-08 19:56:10,902 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 19:56:10,904 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 19:56:10,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 19:56:10,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 19:56:10,904 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-08 19:56:10,904 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:10,904 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:56:10,904 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-08 19:56:10,904 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:56:10,905 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:10,905 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-08 19:56:10,905 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:56:10,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:10,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:56:10,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:56:10,905 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-08 19:56:10,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:56:10,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:56:10,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 19:56:10,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:10,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:56:10,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 19:56:10,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:10,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:56:10,907 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-08 19:56:10,907 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-08 19:56:10,907 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:56:10,907 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:56:10,907 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-08 19:56:10,907 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:56:10,909 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:10,909 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:10,909 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:56:10,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:10,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:10,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:56:10,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:56:10,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:10,909 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:10,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:56:10,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:56:10,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:56:10,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:10,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 19:56:10,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:56:10,910 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:10,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 19:56:10,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:10,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:56:10,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 19:56:10,912 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:10,912 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 19:56:10,912 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:56:10,912 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:56:10,912 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 19:56:10,912 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:56:10,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:56:10,912 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-08 19:56:10,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:56:10,912 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:56:10,912 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-08 19:56:10,912 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:10,912 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 19:56:10,912 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 19:56:10,912 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:10,912 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-08 19:56:10,912 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:56:10,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:56:10,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:56:11,013 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:56:11,013 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:56:11,014 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 19:56:11,015 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 19:56:11,015 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 19:56:11,017 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:56:11,018 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-08 19:56:11,018 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-08 19:56:11,018 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-08 19:56:11,018 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:56:11,018 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-08 19:56:11,018 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-08 19:56:11,019 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:11,019 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 19:56:11,021 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:11,021 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:56:11,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:56:11,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:56:11,022 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:11,022 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:56:11,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 19:56:11,022 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:11,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:56:11,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 19:56:11,023 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-08 19:56:11,023 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-08 19:56:11,023 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-08 19:56:11,023 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-08 19:56:11,023 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:56:11,023 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-08 19:56:11,023 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9. 2023-03-08 19:56:11,023 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9. started... 2023-03-08 19:56:11,023 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811. 2023-03-08 19:56:11,023 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811. started... 2023-03-08 19:56:11,023 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0abcf5c9f041342bc11db8ee9ff848c9 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-08 19:56:11,023 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7. 2023-03-08 19:56:11,023 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 78278c90009823fdb2b2cd7994a08811 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-08 19:56:11,023 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 19:56:11,024 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:56:11,023 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7. started... 2023-03-08 19:56:11,024 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 7becb875ae4303bd834b15dde14376f7 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-08 19:56:11,470 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023030814d3571b5cd246389fc73755dd60229a to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023030814d3571b5cd246389fc73755dd60229a 2023-03-08 19:56:11,486 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308e4ba1f4ec9ce489683cb4c69cad942d8 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230308e4ba1f4ec9ce489683cb4c69cad942d8 2023-03-08 19:56:11,487 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303085294a5359a0e474c839e398c59b5b936 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303085294a5359a0e474c839e398c59b5b936 2023-03-08 19:56:11,507 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/.tmp/family1/6734c404bc5e409fb1fc01a539f45240 2023-03-08 19:56:11,510 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/.tmp/family1/14f12404d9da414ab5341469b81bb072 2023-03-08 19:56:11,564 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308d58c94cb2ed64ce6841494565e86c247 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230308d58c94cb2ed64ce6841494565e86c247 2023-03-08 19:56:11,565 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308000502f523ae4bc3a21e57444eafe85d to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230308000502f523ae4bc3a21e57444eafe85d 2023-03-08 19:56:11,569 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/.tmp/family2/8a024e2ef1594eefb365a8239e28b3d6 2023-03-08 19:56:11,572 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/.tmp/family1/14f12404d9da414ab5341469b81bb072 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family1/14f12404d9da414ab5341469b81bb072 2023-03-08 19:56:11,597 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family1/14f12404d9da414ab5341469b81bb072, entries=4, sequenceid=13, filesize=5.5 K 2023-03-08 19:56:11,598 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/.tmp/family2/8a024e2ef1594eefb365a8239e28b3d6 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family2/8a024e2ef1594eefb365a8239e28b3d6 2023-03-08 19:56:11,601 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family2/8a024e2ef1594eefb365a8239e28b3d6, entries=2, sequenceid=13, filesize=5.1 K 2023-03-08 19:56:11,602 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 7becb875ae4303bd834b15dde14376f7 in 578ms, sequenceid=13, compaction requested=true 2023-03-08 19:56:11,602 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 7becb875ae4303bd834b15dde14376f7: 2023-03-08 19:56:11,602 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7. 2023-03-08 19:56:11,602 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 19:56:11,897 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/.tmp/family1/971684d2cf24497e97deb66cc5227b45 2023-03-08 19:56:11,969 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/.tmp/family2/dad4aad250c6472e89e2219081b0cc65 2023-03-08 19:56:11,974 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/.tmp/family1/6734c404bc5e409fb1fc01a539f45240 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family1/6734c404bc5e409fb1fc01a539f45240 2023-03-08 19:56:12,011 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family1/6734c404bc5e409fb1fc01a539f45240, entries=4, sequenceid=13, filesize=5.5 K 2023-03-08 19:56:12,012 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/.tmp/family2/dad4aad250c6472e89e2219081b0cc65 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family2/dad4aad250c6472e89e2219081b0cc65 2023-03-08 19:56:12,016 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family2/dad4aad250c6472e89e2219081b0cc65, entries=2, sequenceid=13, filesize=5.1 K 2023-03-08 19:56:12,016 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 78278c90009823fdb2b2cd7994a08811 in 993ms, sequenceid=13, compaction requested=true 2023-03-08 19:56:12,017 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 78278c90009823fdb2b2cd7994a08811: 2023-03-08 19:56:12,017 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811. 2023-03-08 19:56:12,017 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 19:56:12,211 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-08 19:56:12,330 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308f5372e53b0744024ae9afc1753c9f5b6 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230308f5372e53b0744024ae9afc1753c9f5b6 2023-03-08 19:56:12,351 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 19:56:12,352 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0bf75cd7 to 127.0.0.1:56751 2023-03-08 19:56:12,352 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 19:56:12,352 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 19:56:12,352 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1b1d8602 to 127.0.0.1:56751 2023-03-08 19:56:12,352 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 19:56:12,753 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/.tmp/family2/3bb3a6e294574140a84944ba88203b32 2023-03-08 19:56:12,757 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/.tmp/family1/971684d2cf24497e97deb66cc5227b45 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family1/971684d2cf24497e97deb66cc5227b45 2023-03-08 19:56:12,779 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family1/971684d2cf24497e97deb66cc5227b45, entries=4, sequenceid=13, filesize=5.5 K 2023-03-08 19:56:12,780 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/.tmp/family2/3bb3a6e294574140a84944ba88203b32 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family2/3bb3a6e294574140a84944ba88203b32 2023-03-08 19:56:12,783 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family2/3bb3a6e294574140a84944ba88203b32, entries=2, sequenceid=13, filesize=5.1 K 2023-03-08 19:56:12,784 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 0abcf5c9f041342bc11db8ee9ff848c9 in 1761ms, sequenceid=13, compaction requested=true 2023-03-08 19:56:12,784 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0abcf5c9f041342bc11db8ee9ff848c9: 2023-03-08 19:56:12,784 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9. 2023-03-08 19:56:12,784 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 19:56:12,784 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 19:56:12,784 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:12,784 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-08 19:56:12,784 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-08 19:56:12,786 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 19:56:12,786 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:12,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:12,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:56:12,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:56:12,786 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 19:56:12,786 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:56:12,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:56:12,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:56:12,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 19:56:12,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:12,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:56:12,787 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-08 19:56:12,787 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@747a63bb[Count = 0] remaining members to acquire global barrier 2023-03-08 19:56:12,787 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-08 19:56:12,787 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 19:56:12,790 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 19:56:12,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 19:56:12,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 19:56:12,790 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-08 19:56:12,790 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-08 19:56:12,790 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:56:12,790 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:12,790 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:56:12,791 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:12,791 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-08 19:56:12,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:12,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:56:12,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:56:12,791 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:56:12,791 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-08 19:56:12,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:56:12,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:56:12,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 19:56:12,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:12,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:56:12,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 19:56:12,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:12,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:56:12,794 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-08 19:56:12,794 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-08 19:56:12,794 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:56:12,794 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:56:12,794 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-08 19:56:12,794 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:56:12,795 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:12,795 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:12,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:12,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:12,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:56:12,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:12,795 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:56:12,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:56:12,795 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:12,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:56:12,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:56:12,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:56:12,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:12,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 19:56:12,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:56:12,796 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:12,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 19:56:12,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:12,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:56:12,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 19:56:12,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:12,798 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:12,798 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 19:56:12,798 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 19:56:12,798 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:12,798 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 19:56:12,799 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-08 19:56:12,799 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:56:12,798 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:56:12,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:56:12,799 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-08 19:56:12,799 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:56:12,799 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:56:12,799 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 19:56:12,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:56:12,799 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:12,799 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-08 19:56:12,799 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:56:12,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:56:12,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:56:12,899 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:56:12,900 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:56:12,901 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 19:56:12,901 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 19:56:12,901 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9. with WAL disabled. Data may be lost in the event of a crash. 2023-03-08 19:56:12,904 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:56:12,905 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-08 19:56:12,905 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-08 19:56:12,905 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-08 19:56:12,905 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:56:12,905 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-08 19:56:12,905 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-08 19:56:12,906 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:12,906 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 19:56:12,910 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:12,910 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:56:12,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:56:12,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:56:12,911 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:12,911 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:56:12,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 19:56:12,911 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:12,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:56:12,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 19:56:12,911 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-08 19:56:12,912 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-08 19:56:12,912 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-08 19:56:12,912 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-08 19:56:12,912 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:56:12,912 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-08 19:56:12,912 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9. 2023-03-08 19:56:12,912 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811. 2023-03-08 19:56:12,912 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 19:56:12,912 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7. 2023-03-08 19:56:12,912 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7. started... 2023-03-08 19:56:12,912 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9. started... 2023-03-08 19:56:12,912 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:56:12,912 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811. started... 2023-03-08 19:56:12,912 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0abcf5c9f041342bc11db8ee9ff848c9 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-08 19:56:12,912 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 7becb875ae4303bd834b15dde14376f7 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-08 19:56:12,912 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 78278c90009823fdb2b2cd7994a08811 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-08 19:56:12,956 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303086c486dacc39b40ca869dfe3d685083a2 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303086c486dacc39b40ca869dfe3d685083a2 2023-03-08 19:56:12,960 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308d5738920ce2447bcbde9ba5055ae6d78 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230308d5738920ce2447bcbde9ba5055ae6d78 2023-03-08 19:56:13,356 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303089516075c89f64cd2aa0d3f06869b32cc to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303089516075c89f64cd2aa0d3f06869b32cc 2023-03-08 19:56:13,373 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/.tmp/family1/fc32f483545c443aa4c4075bdda35a87 2023-03-08 19:56:13,375 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/.tmp/family1/2efbbdc28e2447afbcef6b3ef2cdb2df 2023-03-08 19:56:13,771 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/.tmp/family1/cd5cde866eda46d198eb78783f6ce6b9 2023-03-08 19:56:13,810 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303080206602448b14fc48db1ad6f7882adb0 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202303080206602448b14fc48db1ad6f7882adb0 2023-03-08 19:56:13,810 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303085111e0444d534c60a293ca2b11b67c22 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202303085111e0444d534c60a293ca2b11b67c22 2023-03-08 19:56:14,203 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308a06aa3489b3a481683da5eddea1f5681 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230308a06aa3489b3a481683da5eddea1f5681 2023-03-08 19:56:14,214 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/.tmp/family2/fc176c07ae984884ab9c5b6af929f950 2023-03-08 19:56:14,214 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/.tmp/family2/7e70189afbb641978e4273c31b316e1b 2023-03-08 19:56:14,218 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/.tmp/family1/2efbbdc28e2447afbcef6b3ef2cdb2df as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family1/2efbbdc28e2447afbcef6b3ef2cdb2df 2023-03-08 19:56:14,219 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/.tmp/family1/fc32f483545c443aa4c4075bdda35a87 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family1/fc32f483545c443aa4c4075bdda35a87 2023-03-08 19:56:14,241 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family1/2efbbdc28e2447afbcef6b3ef2cdb2df, entries=4, sequenceid=17, filesize=5.5 K 2023-03-08 19:56:14,241 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family1/fc32f483545c443aa4c4075bdda35a87, entries=4, sequenceid=17, filesize=5.5 K 2023-03-08 19:56:14,242 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/.tmp/family2/fc176c07ae984884ab9c5b6af929f950 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family2/fc176c07ae984884ab9c5b6af929f950 2023-03-08 19:56:14,242 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/.tmp/family2/7e70189afbb641978e4273c31b316e1b as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family2/7e70189afbb641978e4273c31b316e1b 2023-03-08 19:56:14,246 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family2/7e70189afbb641978e4273c31b316e1b, entries=2, sequenceid=17, filesize=5.1 K 2023-03-08 19:56:14,246 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family2/fc176c07ae984884ab9c5b6af929f950, entries=2, sequenceid=17, filesize=5.1 K 2023-03-08 19:56:14,247 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 78278c90009823fdb2b2cd7994a08811 in 1334ms, sequenceid=17, compaction requested=true 2023-03-08 19:56:14,247 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 0abcf5c9f041342bc11db8ee9ff848c9 in 1335ms, sequenceid=17, compaction requested=true 2023-03-08 19:56:14,247 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 78278c90009823fdb2b2cd7994a08811: 2023-03-08 19:56:14,247 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0abcf5c9f041342bc11db8ee9ff848c9: 2023-03-08 19:56:14,247 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811. 2023-03-08 19:56:14,247 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9. 2023-03-08 19:56:14,247 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 19:56:14,247 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 19:56:14,609 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/.tmp/family2/143ae82089674f298f04a17d361ec4f5 2023-03-08 19:56:14,613 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/.tmp/family1/cd5cde866eda46d198eb78783f6ce6b9 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family1/cd5cde866eda46d198eb78783f6ce6b9 2023-03-08 19:56:14,639 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family1/cd5cde866eda46d198eb78783f6ce6b9, entries=4, sequenceid=17, filesize=5.5 K 2023-03-08 19:56:14,639 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/.tmp/family2/143ae82089674f298f04a17d361ec4f5 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family2/143ae82089674f298f04a17d361ec4f5 2023-03-08 19:56:14,644 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family2/143ae82089674f298f04a17d361ec4f5, entries=2, sequenceid=17, filesize=5.1 K 2023-03-08 19:56:14,645 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 7becb875ae4303bd834b15dde14376f7 in 1733ms, sequenceid=17, compaction requested=true 2023-03-08 19:56:14,645 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 7becb875ae4303bd834b15dde14376f7: 2023-03-08 19:56:14,645 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7. 2023-03-08 19:56:14,645 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 19:56:14,645 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 19:56:14,645 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:14,645 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-08 19:56:14,645 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-08 19:56:14,647 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:14,647 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 19:56:14,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:14,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:56:14,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:56:14,647 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 19:56:14,647 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:56:14,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:56:14,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:56:14,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 19:56:14,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:14,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:56:14,649 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-08 19:56:14,649 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7ad07229[Count = 0] remaining members to acquire global barrier 2023-03-08 19:56:14,649 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-08 19:56:14,649 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 19:56:14,650 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 19:56:14,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 19:56:14,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 19:56:14,650 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-08 19:56:14,650 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-08 19:56:14,650 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:56:14,650 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:14,650 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:56:14,652 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-08 19:56:14,652 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:14,652 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:56:14,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:14,652 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-08 19:56:14,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:56:14,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:56:14,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:56:14,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:56:14,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 19:56:14,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:14,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:56:14,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 19:56:14,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:14,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:56:14,654 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-08 19:56:14,654 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-08 19:56:14,654 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:56:14,654 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:56:14,654 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-08 19:56:14,654 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:56:14,656 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:14,656 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:14,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:14,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:14,656 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:56:14,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:14,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:56:14,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:56:14,657 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:56:14,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:56:14,657 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:14,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:56:14,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:14,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 19:56:14,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:56:14,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 19:56:14,658 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:14,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:14,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:56:14,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 19:56:14,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:14,661 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:56:14,661 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:14,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:56:14,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:56:14,661 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 19:56:14,661 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:56:14,661 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 19:56:14,661 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-08 19:56:14,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:56:14,661 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:56:14,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:56:14,662 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:56:14,662 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:14,662 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 19:56:14,662 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 19:56:14,662 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:14,662 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-08 19:56:14,662 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:56:14,762 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:56:14,763 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:56:14,772 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 19:56:14,779 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 19:56:14,785 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 19:56:14,786 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 19:56:14,888 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-08 19:56:14,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-08 19:56:14,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-08 19:56:14,890 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-08 19:56:14,890 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:56:14,890 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-08 19:56:14,890 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-08 19:56:14,891 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:14,891 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 19:56:14,892 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:56:14,892 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:14,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:56:14,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:56:14,892 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:14,892 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-08 19:56:14,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 19:56:14,893 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:14,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-08 19:56:14,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 19:56:14,893 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-08 19:56:14,896 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-08 19:56:14,896 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-08 19:56:14,896 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-08 19:56:14,896 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-08 19:56:14,896 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-08 19:56:14,896 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-08 19:56:14,896 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-08 19:56:14,896 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9. 2023-03-08 19:56:14,896 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811. 2023-03-08 19:56:14,896 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7. 2023-03-08 19:56:14,896 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9. started... 2023-03-08 19:56:14,897 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7. started... 2023-03-08 19:56:14,897 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811. started... 2023-03-08 19:56:14,897 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0abcf5c9f041342bc11db8ee9ff848c9 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-08 19:56:14,897 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 7becb875ae4303bd834b15dde14376f7 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-08 19:56:14,897 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 78278c90009823fdb2b2cd7994a08811 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-08 19:56:14,943 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/.tmp/family1/8709c017f919438b8d663d6f02262d07 2023-03-08 19:56:14,969 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8709c017f919438b8d663d6f02262d07 2023-03-08 19:56:15,341 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/.tmp/family1/3ff4a9e476c640a09f86662c1892e82e 2023-03-08 19:56:15,353 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/.tmp/family1/19b2433c320a4ae688d6c98a3d083c54 2023-03-08 19:56:15,375 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 19b2433c320a4ae688d6c98a3d083c54 2023-03-08 19:56:15,376 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3ff4a9e476c640a09f86662c1892e82e 2023-03-08 19:56:15,382 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/.tmp/family2/93f028db7ee54d548ce040ead4373c4f 2023-03-08 19:56:15,385 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 93f028db7ee54d548ce040ead4373c4f 2023-03-08 19:56:15,386 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/.tmp/family2/22cc338157254a8bb1a7f9b300089bbe 2023-03-08 19:56:15,386 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/.tmp/family1/8709c017f919438b8d663d6f02262d07 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family1/8709c017f919438b8d663d6f02262d07 2023-03-08 19:56:15,388 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/.tmp/family2/7edd009624e64724bb94ccdd1b41c1b3 2023-03-08 19:56:15,389 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 22cc338157254a8bb1a7f9b300089bbe 2023-03-08 19:56:15,390 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/.tmp/family1/19b2433c320a4ae688d6c98a3d083c54 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family1/19b2433c320a4ae688d6c98a3d083c54 2023-03-08 19:56:15,391 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7edd009624e64724bb94ccdd1b41c1b3 2023-03-08 19:56:15,392 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/.tmp/family1/3ff4a9e476c640a09f86662c1892e82e as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family1/3ff4a9e476c640a09f86662c1892e82e 2023-03-08 19:56:15,411 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8709c017f919438b8d663d6f02262d07 2023-03-08 19:56:15,411 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family1/8709c017f919438b8d663d6f02262d07, entries=3, sequenceid=23, filesize=5.2 K 2023-03-08 19:56:15,411 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/.tmp/family2/93f028db7ee54d548ce040ead4373c4f as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family2/93f028db7ee54d548ce040ead4373c4f 2023-03-08 19:56:15,412 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 19b2433c320a4ae688d6c98a3d083c54 2023-03-08 19:56:15,412 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family1/19b2433c320a4ae688d6c98a3d083c54, entries=3, sequenceid=23, filesize=5.2 K 2023-03-08 19:56:15,413 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/.tmp/family2/22cc338157254a8bb1a7f9b300089bbe as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family2/22cc338157254a8bb1a7f9b300089bbe 2023-03-08 19:56:15,414 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3ff4a9e476c640a09f86662c1892e82e 2023-03-08 19:56:15,414 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family1/3ff4a9e476c640a09f86662c1892e82e, entries=3, sequenceid=23, filesize=5.2 K 2023-03-08 19:56:15,415 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/.tmp/family2/7edd009624e64724bb94ccdd1b41c1b3 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family2/7edd009624e64724bb94ccdd1b41c1b3 2023-03-08 19:56:15,416 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 93f028db7ee54d548ce040ead4373c4f 2023-03-08 19:56:15,416 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family2/93f028db7ee54d548ce040ead4373c4f, entries=1, sequenceid=23, filesize=5.0 K 2023-03-08 19:56:15,416 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 22cc338157254a8bb1a7f9b300089bbe 2023-03-08 19:56:15,416 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family2/22cc338157254a8bb1a7f9b300089bbe, entries=1, sequenceid=23, filesize=5.0 K 2023-03-08 19:56:15,417 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 78278c90009823fdb2b2cd7994a08811 in 520ms, sequenceid=23, compaction requested=true 2023-03-08 19:56:15,417 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 0abcf5c9f041342bc11db8ee9ff848c9 in 520ms, sequenceid=23, compaction requested=true 2023-03-08 19:56:15,417 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 78278c90009823fdb2b2cd7994a08811: 2023-03-08 19:56:15,417 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0abcf5c9f041342bc11db8ee9ff848c9: 2023-03-08 19:56:15,417 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811. 2023-03-08 19:56:15,417 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9. 2023-03-08 19:56:15,417 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-08 19:56:15,417 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-08 19:56:15,419 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7edd009624e64724bb94ccdd1b41c1b3 2023-03-08 19:56:15,419 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family2/7edd009624e64724bb94ccdd1b41c1b3, entries=1, sequenceid=23, filesize=5.0 K 2023-03-08 19:56:15,419 INFO [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 7becb875ae4303bd834b15dde14376f7 in 522ms, sequenceid=23, compaction requested=true 2023-03-08 19:56:15,419 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 7becb875ae4303bd834b15dde14376f7: 2023-03-08 19:56:15,419 DEBUG [rs(jenkins-hbase4.apache.org,33367,1678305218680)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7. 2023-03-08 19:56:15,419 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-08 19:56:15,419 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-08 19:56:15,419 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:15,419 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-08 19:56:15,420 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-08 19:56:15,421 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:15,421 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 19:56:15,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:15,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:56:15,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:56:15,421 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 19:56:15,421 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-08 19:56:15,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:56:15,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:56:15,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 19:56:15,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:15,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:56:15,423 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33367,1678305218680' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-08 19:56:15,423 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3b1fd8fa[Count = 0] remaining members to acquire global barrier 2023-03-08 19:56:15,423 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-08 19:56:15,423 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 19:56:15,424 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 19:56:15,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 19:56:15,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 19:56:15,424 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-08 19:56:15,424 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-08 19:56:15,424 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,33367,1678305218680' in zk 2023-03-08 19:56:15,424 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:15,424 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-08 19:56:15,425 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-08 19:56:15,425 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:15,425 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:56:15,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:15,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:56:15,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:56:15,425 DEBUG [member: 'jenkins-hbase4.apache.org,33367,1678305218680' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-08 19:56:15,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:56:15,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:56:15,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 19:56:15,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:15,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:56:15,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 19:56:15,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:15,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,33367,1678305218680': 2023-03-08 19:56:15,428 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33367,1678305218680' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-08 19:56:15,428 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-08 19:56:15,428 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-08 19:56:15,428 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-08 19:56:15,428 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-08 19:56:15,428 INFO [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-08 19:56:15,430 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:15,430 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:15,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:15,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:15,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-08 19:56:15,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-08 19:56:15,430 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:56:15,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:15,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:56:15,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:56:15,430 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:15,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-08 19:56:15,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:15,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 19:56:15,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-08 19:56:15,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 19:56:15,432 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:15,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:15,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-08 19:56:15,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-08 19:56:15,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:15,434 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:15,434 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 19:56:15,434 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-08 19:56:15,434 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-08 19:56:15,434 DEBUG [(jenkins-hbase4.apache.org,39989,1678305217508)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-08 19:56:15,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-08 19:56:15,434 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:15,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-08 19:56:15,434 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 19:56:15,434 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-08 19:56:15,434 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-08 19:56:15,434 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-08 19:56:15,434 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-08 19:56:15,434 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-08 19:56:15,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-08 19:56:15,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-08 19:56:15,434 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-08 19:56:15,434 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-08 19:56:15,534 DEBUG [Listener at localhost/34049] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-08 19:56:15,535 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39989] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-08 19:56:15,536 DEBUG [Listener at localhost/34049] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 19:56:15,536 DEBUG [Listener at localhost/34049] regionserver.HStore(1912): 0abcf5c9f041342bc11db8ee9ff848c9/family1 is initiating major compaction (all files) 2023-03-08 19:56:15,536 INFO [Listener at localhost/34049] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 19:56:15,536 INFO [Listener at localhost/34049] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 19:56:15,536 INFO [Listener at localhost/34049] regionserver.HRegion(2259): Starting compaction of 0abcf5c9f041342bc11db8ee9ff848c9/family1 in testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9. 2023-03-08 19:56:15,536 INFO [Listener at localhost/34049] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family1/57923c96e6164f0a9ad738427b68febd, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family1/62830d7c5b024fcd8575068c5b33cabf, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family1/971684d2cf24497e97deb66cc5227b45, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family1/2efbbdc28e2447afbcef6b3ef2cdb2df, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family1/19b2433c320a4ae688d6c98a3d083c54] into tmpdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/.tmp, totalSize=27.1 K 2023-03-08 19:56:15,537 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 57923c96e6164f0a9ad738427b68febd, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678305367683 2023-03-08 19:56:15,537 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 62830d7c5b024fcd8575068c5b33cabf, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678305369165 2023-03-08 19:56:15,537 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 971684d2cf24497e97deb66cc5227b45, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678305371015 2023-03-08 19:56:15,538 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 2efbbdc28e2447afbcef6b3ef2cdb2df, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678305372901 2023-03-08 19:56:15,538 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 19b2433c320a4ae688d6c98a3d083c54, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 19:56:15,548 INFO [Listener at localhost/34049] throttle.PressureAwareThroughputController(145): 0abcf5c9f041342bc11db8ee9ff848c9#family1#compaction#198 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 19:56:15,986 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202303088c307aba4efd41a799172c75c8c186e9_del 2023-03-08 19:56:15,987 INFO [Listener at localhost/34049] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303088c307aba4efd41a799172c75c8c186e9_del to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303088c307aba4efd41a799172c75c8c186e9_del 2023-03-08 19:56:16,839 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8d7beccdb3b64663a713014af777b365 2023-03-08 19:56:16,839 DEBUG [Listener at localhost/34049] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/.tmp/family1/8d7beccdb3b64663a713014af777b365 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family1/8d7beccdb3b64663a713014af777b365 2023-03-08 19:56:16,862 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8d7beccdb3b64663a713014af777b365 2023-03-08 19:56:16,863 INFO [Listener at localhost/34049] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 0abcf5c9f041342bc11db8ee9ff848c9/family1 of 0abcf5c9f041342bc11db8ee9ff848c9 into 8d7beccdb3b64663a713014af777b365(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 19:56:16,863 DEBUG [Listener at localhost/34049] regionserver.HRegion(2289): Compaction status journal for 0abcf5c9f041342bc11db8ee9ff848c9: 2023-03-08 19:56:16,863 DEBUG [Listener at localhost/34049] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 19:56:16,863 DEBUG [Listener at localhost/34049] regionserver.HStore(1912): 0abcf5c9f041342bc11db8ee9ff848c9/family2 is initiating major compaction (all files) 2023-03-08 19:56:16,863 INFO [Listener at localhost/34049] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 19:56:16,863 INFO [Listener at localhost/34049] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 19:56:16,863 INFO [Listener at localhost/34049] regionserver.HRegion(2259): Starting compaction of 0abcf5c9f041342bc11db8ee9ff848c9/family2 in testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9. 2023-03-08 19:56:16,863 INFO [Listener at localhost/34049] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family2/f47a02760b4f4175bec133816380b0c1, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family2/0f6a9c3780cc49fd8eecb5bffdefd4a9, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family2/3bb3a6e294574140a84944ba88203b32, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family2/fc176c07ae984884ab9c5b6af929f950, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family2/22cc338157254a8bb1a7f9b300089bbe] into tmpdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/.tmp, totalSize=25.4 K 2023-03-08 19:56:16,864 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting f47a02760b4f4175bec133816380b0c1, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678305367683 2023-03-08 19:56:16,864 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 0f6a9c3780cc49fd8eecb5bffdefd4a9, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678305369165 2023-03-08 19:56:16,864 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 3bb3a6e294574140a84944ba88203b32, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678305371015 2023-03-08 19:56:16,865 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting fc176c07ae984884ab9c5b6af929f950, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678305372901 2023-03-08 19:56:16,865 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 22cc338157254a8bb1a7f9b300089bbe, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 19:56:16,872 INFO [Listener at localhost/34049] throttle.PressureAwareThroughputController(145): 0abcf5c9f041342bc11db8ee9ff848c9#family2#compaction#199 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 19:56:16,897 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family1/57923c96e6164f0a9ad738427b68febd, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family1/62830d7c5b024fcd8575068c5b33cabf, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family1/971684d2cf24497e97deb66cc5227b45, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family1/2efbbdc28e2447afbcef6b3ef2cdb2df, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family1/19b2433c320a4ae688d6c98a3d083c54] to archive 2023-03-08 19:56:16,898 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:56:16,899 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family1/57923c96e6164f0a9ad738427b68febd to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family1/57923c96e6164f0a9ad738427b68febd 2023-03-08 19:56:16,900 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family1/62830d7c5b024fcd8575068c5b33cabf to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family1/62830d7c5b024fcd8575068c5b33cabf 2023-03-08 19:56:16,901 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family1/971684d2cf24497e97deb66cc5227b45 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family1/971684d2cf24497e97deb66cc5227b45 2023-03-08 19:56:16,902 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family1/2efbbdc28e2447afbcef6b3ef2cdb2df to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family1/2efbbdc28e2447afbcef6b3ef2cdb2df 2023-03-08 19:56:16,903 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family1/19b2433c320a4ae688d6c98a3d083c54 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family1/19b2433c320a4ae688d6c98a3d083c54 2023-03-08 19:56:17,280 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202303083e536f30252a4006a37287ad1eaf7c49_del 2023-03-08 19:56:17,281 INFO [Listener at localhost/34049] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303083e536f30252a4006a37287ad1eaf7c49_del to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202303083e536f30252a4006a37287ad1eaf7c49_del 2023-03-08 19:56:18,093 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c6f55b4921664688a224835ffab57d25 2023-03-08 19:56:18,094 DEBUG [Listener at localhost/34049] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/.tmp/family2/c6f55b4921664688a224835ffab57d25 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family2/c6f55b4921664688a224835ffab57d25 2023-03-08 19:56:18,098 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c6f55b4921664688a224835ffab57d25 2023-03-08 19:56:18,098 INFO [Listener at localhost/34049] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 0abcf5c9f041342bc11db8ee9ff848c9/family2 of 0abcf5c9f041342bc11db8ee9ff848c9 into c6f55b4921664688a224835ffab57d25(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 19:56:18,099 DEBUG [Listener at localhost/34049] regionserver.HRegion(2289): Compaction status journal for 0abcf5c9f041342bc11db8ee9ff848c9: 2023-03-08 19:56:18,099 DEBUG [Listener at localhost/34049] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 19:56:18,099 DEBUG [Listener at localhost/34049] regionserver.HStore(1912): 78278c90009823fdb2b2cd7994a08811/family1 is initiating major compaction (all files) 2023-03-08 19:56:18,100 INFO [Listener at localhost/34049] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 19:56:18,100 INFO [Listener at localhost/34049] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 19:56:18,100 INFO [Listener at localhost/34049] regionserver.HRegion(2259): Starting compaction of 78278c90009823fdb2b2cd7994a08811/family1 in testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811. 2023-03-08 19:56:18,101 INFO [Listener at localhost/34049] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family1/b4a72f07e25c4cd1b4b80a9c7684653d, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family1/b7d8bbfffb884af7a33b742a441b4543, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family1/6734c404bc5e409fb1fc01a539f45240, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family1/fc32f483545c443aa4c4075bdda35a87, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family1/8709c017f919438b8d663d6f02262d07] into tmpdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/.tmp, totalSize=27.1 K 2023-03-08 19:56:18,101 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting b4a72f07e25c4cd1b4b80a9c7684653d, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678305367682 2023-03-08 19:56:18,101 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting b7d8bbfffb884af7a33b742a441b4543, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678305369164 2023-03-08 19:56:18,102 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 6734c404bc5e409fb1fc01a539f45240, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678305371014 2023-03-08 19:56:18,102 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting fc32f483545c443aa4c4075bdda35a87, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678305372901 2023-03-08 19:56:18,102 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 8709c017f919438b8d663d6f02262d07, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 19:56:18,111 INFO [Listener at localhost/34049] throttle.PressureAwareThroughputController(145): 78278c90009823fdb2b2cd7994a08811#family1#compaction#200 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 19:56:18,198 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family2/f47a02760b4f4175bec133816380b0c1, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family2/0f6a9c3780cc49fd8eecb5bffdefd4a9, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family2/3bb3a6e294574140a84944ba88203b32, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family2/fc176c07ae984884ab9c5b6af929f950, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family2/22cc338157254a8bb1a7f9b300089bbe] to archive 2023-03-08 19:56:18,199 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:56:18,200 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family2/f47a02760b4f4175bec133816380b0c1 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family2/f47a02760b4f4175bec133816380b0c1 2023-03-08 19:56:18,201 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family2/0f6a9c3780cc49fd8eecb5bffdefd4a9 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family2/0f6a9c3780cc49fd8eecb5bffdefd4a9 2023-03-08 19:56:18,202 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family2/3bb3a6e294574140a84944ba88203b32 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family2/3bb3a6e294574140a84944ba88203b32 2023-03-08 19:56:18,203 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family2/fc176c07ae984884ab9c5b6af929f950 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family2/fc176c07ae984884ab9c5b6af929f950 2023-03-08 19:56:18,204 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family2/22cc338157254a8bb1a7f9b300089bbe to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family2/22cc338157254a8bb1a7f9b300089bbe 2023-03-08 19:56:18,548 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023030864d271fe41764bea9f399565e9001e78_del 2023-03-08 19:56:18,549 INFO [Listener at localhost/34049] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023030864d271fe41764bea9f399565e9001e78_del to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023030864d271fe41764bea9f399565e9001e78_del 2023-03-08 19:56:19,401 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 22717c23eae84f6f9e47f2b5ec37af8f 2023-03-08 19:56:19,402 DEBUG [Listener at localhost/34049] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/.tmp/family1/22717c23eae84f6f9e47f2b5ec37af8f as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family1/22717c23eae84f6f9e47f2b5ec37af8f 2023-03-08 19:56:19,429 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 22717c23eae84f6f9e47f2b5ec37af8f 2023-03-08 19:56:19,430 INFO [Listener at localhost/34049] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 78278c90009823fdb2b2cd7994a08811/family1 of 78278c90009823fdb2b2cd7994a08811 into 22717c23eae84f6f9e47f2b5ec37af8f(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 19:56:19,430 DEBUG [Listener at localhost/34049] regionserver.HRegion(2289): Compaction status journal for 78278c90009823fdb2b2cd7994a08811: 2023-03-08 19:56:19,430 DEBUG [Listener at localhost/34049] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 19:56:19,430 DEBUG [Listener at localhost/34049] regionserver.HStore(1912): 78278c90009823fdb2b2cd7994a08811/family2 is initiating major compaction (all files) 2023-03-08 19:56:19,430 INFO [Listener at localhost/34049] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 19:56:19,430 INFO [Listener at localhost/34049] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 19:56:19,430 INFO [Listener at localhost/34049] regionserver.HRegion(2259): Starting compaction of 78278c90009823fdb2b2cd7994a08811/family2 in testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811. 2023-03-08 19:56:19,431 INFO [Listener at localhost/34049] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family2/8b7079ad98e640428777ff32b0dbab45, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family2/2018f85e8b2244cb84b0f7f433314ebc, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family2/dad4aad250c6472e89e2219081b0cc65, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family2/7e70189afbb641978e4273c31b316e1b, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family2/93f028db7ee54d548ce040ead4373c4f] into tmpdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/.tmp, totalSize=25.4 K 2023-03-08 19:56:19,431 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 8b7079ad98e640428777ff32b0dbab45, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678305367682 2023-03-08 19:56:19,431 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 2018f85e8b2244cb84b0f7f433314ebc, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678305369164 2023-03-08 19:56:19,432 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting dad4aad250c6472e89e2219081b0cc65, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678305371014 2023-03-08 19:56:19,432 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 7e70189afbb641978e4273c31b316e1b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678305372901 2023-03-08 19:56:19,432 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 93f028db7ee54d548ce040ead4373c4f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 19:56:19,440 INFO [Listener at localhost/34049] throttle.PressureAwareThroughputController(145): 78278c90009823fdb2b2cd7994a08811#family2#compaction#201 average throughput is 1.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 19:56:19,501 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family1/b4a72f07e25c4cd1b4b80a9c7684653d, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family1/b7d8bbfffb884af7a33b742a441b4543, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family1/6734c404bc5e409fb1fc01a539f45240, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family1/fc32f483545c443aa4c4075bdda35a87, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family1/8709c017f919438b8d663d6f02262d07] to archive 2023-03-08 19:56:19,502 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:56:19,504 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family1/b4a72f07e25c4cd1b4b80a9c7684653d to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family1/b4a72f07e25c4cd1b4b80a9c7684653d 2023-03-08 19:56:19,505 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family1/b7d8bbfffb884af7a33b742a441b4543 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family1/b7d8bbfffb884af7a33b742a441b4543 2023-03-08 19:56:19,506 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family1/6734c404bc5e409fb1fc01a539f45240 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family1/6734c404bc5e409fb1fc01a539f45240 2023-03-08 19:56:19,507 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family1/fc32f483545c443aa4c4075bdda35a87 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family1/fc32f483545c443aa4c4075bdda35a87 2023-03-08 19:56:19,508 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family1/8709c017f919438b8d663d6f02262d07 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family1/8709c017f919438b8d663d6f02262d07 2023-03-08 19:56:19,849 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230308507c8606193c4dbf8a0c0cca5f796af9_del 2023-03-08 19:56:19,849 INFO [Listener at localhost/34049] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230308507c8606193c4dbf8a0c0cca5f796af9_del to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230308507c8606193c4dbf8a0c0cca5f796af9_del 2023-03-08 19:56:20,664 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0cffbd386a874ed5967539044ff3aeb1 2023-03-08 19:56:20,665 DEBUG [Listener at localhost/34049] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/.tmp/family2/0cffbd386a874ed5967539044ff3aeb1 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family2/0cffbd386a874ed5967539044ff3aeb1 2023-03-08 19:56:20,670 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0cffbd386a874ed5967539044ff3aeb1 2023-03-08 19:56:20,671 INFO [Listener at localhost/34049] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 78278c90009823fdb2b2cd7994a08811/family2 of 78278c90009823fdb2b2cd7994a08811 into 0cffbd386a874ed5967539044ff3aeb1(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 19:56:20,671 DEBUG [Listener at localhost/34049] regionserver.HRegion(2289): Compaction status journal for 78278c90009823fdb2b2cd7994a08811: 2023-03-08 19:56:20,671 DEBUG [Listener at localhost/34049] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 19:56:20,671 DEBUG [Listener at localhost/34049] regionserver.HStore(1912): 7becb875ae4303bd834b15dde14376f7/family1 is initiating major compaction (all files) 2023-03-08 19:56:20,671 INFO [Listener at localhost/34049] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 19:56:20,671 INFO [Listener at localhost/34049] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 19:56:20,671 INFO [Listener at localhost/34049] regionserver.HRegion(2259): Starting compaction of 7becb875ae4303bd834b15dde14376f7/family1 in testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7. 2023-03-08 19:56:20,671 INFO [Listener at localhost/34049] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family1/3df95a65498e45a4841c4a31b6a10750, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family1/0ecd44f886594522b75e5da77e30afa3, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family1/14f12404d9da414ab5341469b81bb072, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family1/cd5cde866eda46d198eb78783f6ce6b9, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family1/3ff4a9e476c640a09f86662c1892e82e] into tmpdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/.tmp, totalSize=27.1 K 2023-03-08 19:56:20,672 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 3df95a65498e45a4841c4a31b6a10750, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678305367683 2023-03-08 19:56:20,672 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 0ecd44f886594522b75e5da77e30afa3, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678305369164 2023-03-08 19:56:20,672 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 14f12404d9da414ab5341469b81bb072, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678305371015 2023-03-08 19:56:20,673 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting cd5cde866eda46d198eb78783f6ce6b9, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678305372901 2023-03-08 19:56:20,673 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 3ff4a9e476c640a09f86662c1892e82e, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 19:56:20,682 INFO [Listener at localhost/34049] throttle.PressureAwareThroughputController(145): 7becb875ae4303bd834b15dde14376f7#family1#compaction#202 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 19:56:20,698 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family2/8b7079ad98e640428777ff32b0dbab45, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family2/2018f85e8b2244cb84b0f7f433314ebc, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family2/dad4aad250c6472e89e2219081b0cc65, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family2/7e70189afbb641978e4273c31b316e1b, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family2/93f028db7ee54d548ce040ead4373c4f] to archive 2023-03-08 19:56:20,698 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:56:20,699 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family2/8b7079ad98e640428777ff32b0dbab45 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family2/8b7079ad98e640428777ff32b0dbab45 2023-03-08 19:56:20,700 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family2/2018f85e8b2244cb84b0f7f433314ebc to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family2/2018f85e8b2244cb84b0f7f433314ebc 2023-03-08 19:56:20,701 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family2/dad4aad250c6472e89e2219081b0cc65 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family2/dad4aad250c6472e89e2219081b0cc65 2023-03-08 19:56:20,702 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family2/7e70189afbb641978e4273c31b316e1b to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family2/7e70189afbb641978e4273c31b316e1b 2023-03-08 19:56:20,703 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family2/93f028db7ee54d548ce040ead4373c4f to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family2/93f028db7ee54d548ce040ead4373c4f 2023-03-08 19:56:21,121 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230308e05246412da04adda61dfa47e61b1e20_del 2023-03-08 19:56:21,121 INFO [Listener at localhost/34049] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230308e05246412da04adda61dfa47e61b1e20_del to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230308e05246412da04adda61dfa47e61b1e20_del 2023-03-08 19:56:21,976 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cde4d00f97754d4eb39bc097015cdd71 2023-03-08 19:56:21,977 DEBUG [Listener at localhost/34049] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/.tmp/family1/cde4d00f97754d4eb39bc097015cdd71 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family1/cde4d00f97754d4eb39bc097015cdd71 2023-03-08 19:56:22,000 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cde4d00f97754d4eb39bc097015cdd71 2023-03-08 19:56:22,001 INFO [Listener at localhost/34049] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 7becb875ae4303bd834b15dde14376f7/family1 of 7becb875ae4303bd834b15dde14376f7 into cde4d00f97754d4eb39bc097015cdd71(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-08 19:56:22,001 DEBUG [Listener at localhost/34049] regionserver.HRegion(2289): Compaction status journal for 7becb875ae4303bd834b15dde14376f7: 2023-03-08 19:56:22,001 DEBUG [Listener at localhost/34049] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-08 19:56:22,001 DEBUG [Listener at localhost/34049] regionserver.HStore(1912): 7becb875ae4303bd834b15dde14376f7/family2 is initiating major compaction (all files) 2023-03-08 19:56:22,001 INFO [Listener at localhost/34049] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-08 19:56:22,001 INFO [Listener at localhost/34049] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-08 19:56:22,002 INFO [Listener at localhost/34049] regionserver.HRegion(2259): Starting compaction of 7becb875ae4303bd834b15dde14376f7/family2 in testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7. 2023-03-08 19:56:22,002 INFO [Listener at localhost/34049] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family2/107493347ac944efa7a27877fd5ec384, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family2/a9804969245048d995d1e495e20df9ce, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family2/8a024e2ef1594eefb365a8239e28b3d6, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family2/143ae82089674f298f04a17d361ec4f5, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family2/7edd009624e64724bb94ccdd1b41c1b3] into tmpdir=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/.tmp, totalSize=25.4 K 2023-03-08 19:56:22,002 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 107493347ac944efa7a27877fd5ec384, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678305367683 2023-03-08 19:56:22,002 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting a9804969245048d995d1e495e20df9ce, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678305369164 2023-03-08 19:56:22,003 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 8a024e2ef1594eefb365a8239e28b3d6, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678305371015 2023-03-08 19:56:22,003 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 143ae82089674f298f04a17d361ec4f5, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678305372901 2023-03-08 19:56:22,003 DEBUG [Listener at localhost/34049] compactions.Compactor(207): Compacting 7edd009624e64724bb94ccdd1b41c1b3, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-08 19:56:22,011 INFO [Listener at localhost/34049] throttle.PressureAwareThroughputController(145): 7becb875ae4303bd834b15dde14376f7#family2#compaction#203 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-08 19:56:22,104 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family1/3df95a65498e45a4841c4a31b6a10750, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family1/0ecd44f886594522b75e5da77e30afa3, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family1/14f12404d9da414ab5341469b81bb072, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family1/cd5cde866eda46d198eb78783f6ce6b9, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family1/3ff4a9e476c640a09f86662c1892e82e] to archive 2023-03-08 19:56:22,105 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:56:22,106 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family1/3df95a65498e45a4841c4a31b6a10750 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family1/3df95a65498e45a4841c4a31b6a10750 2023-03-08 19:56:22,107 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family1/0ecd44f886594522b75e5da77e30afa3 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family1/0ecd44f886594522b75e5da77e30afa3 2023-03-08 19:56:22,108 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family1/14f12404d9da414ab5341469b81bb072 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family1/14f12404d9da414ab5341469b81bb072 2023-03-08 19:56:22,109 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family1/cd5cde866eda46d198eb78783f6ce6b9 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family1/cd5cde866eda46d198eb78783f6ce6b9 2023-03-08 19:56:22,111 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family1/3ff4a9e476c640a09f86662c1892e82e to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family1/3ff4a9e476c640a09f86662c1892e82e 2023-03-08 19:56:22,418 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202303086559023370b84d1c9e8ba7d5b37fb036_del 2023-03-08 19:56:22,418 INFO [Listener at localhost/34049] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303086559023370b84d1c9e8ba7d5b37fb036_del to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b202303086559023370b84d1c9e8ba7d5b37fb036_del 2023-03-08 19:56:22,831 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 40457863126c4e468d9b3ed805f8de53 2023-03-08 19:56:22,831 DEBUG [Listener at localhost/34049] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/.tmp/family2/40457863126c4e468d9b3ed805f8de53 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family2/40457863126c4e468d9b3ed805f8de53 2023-03-08 19:56:22,835 INFO [Listener at localhost/34049] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 40457863126c4e468d9b3ed805f8de53 2023-03-08 19:56:22,836 INFO [Listener at localhost/34049] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 7becb875ae4303bd834b15dde14376f7/family2 of 7becb875ae4303bd834b15dde14376f7 into 40457863126c4e468d9b3ed805f8de53(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-08 19:56:22,836 DEBUG [Listener at localhost/34049] regionserver.HRegion(2289): Compaction status journal for 7becb875ae4303bd834b15dde14376f7: 2023-03-08 19:56:22,839 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 19:56:22,841 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 19:56:22,845 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 19:56:22,846 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 19:56:22,851 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39989] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xBDx\xE3<\x98\xBF\xFAP\xED\xDC\xC5\xDCr\x1D\xF4\x11"\x10pG-\xB6\x80~\xD7\xD6\xCCw\xCEC\x03\x15\xE8\xB3*\x10\xE1\xB9\x82+9\x12$eX3\xC9n\x07-\xE2\x002\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testMajorCompactionFromAdmin 2023-03-08 19:56:22,873 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] procedure2.ProcedureExecutor(1029): Stored pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-03-08 19:56:22,874 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-03-08 19:56:22,874 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 19:56:22,874 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=123, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678305982874 2023-03-08 19:56:22,875 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-08 19:56:22,898 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family2/107493347ac944efa7a27877fd5ec384, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family2/a9804969245048d995d1e495e20df9ce, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family2/8a024e2ef1594eefb365a8239e28b3d6, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family2/143ae82089674f298f04a17d361ec4f5, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family2/7edd009624e64724bb94ccdd1b41c1b3] to archive 2023-03-08 19:56:22,898 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:56:22,900 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family2/107493347ac944efa7a27877fd5ec384 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family2/107493347ac944efa7a27877fd5ec384 2023-03-08 19:56:22,901 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family2/a9804969245048d995d1e495e20df9ce to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family2/a9804969245048d995d1e495e20df9ce 2023-03-08 19:56:22,902 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family2/8a024e2ef1594eefb365a8239e28b3d6 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family2/8a024e2ef1594eefb365a8239e28b3d6 2023-03-08 19:56:22,903 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family2/143ae82089674f298f04a17d361ec4f5 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family2/143ae82089674f298f04a17d361ec4f5 2023-03-08 19:56:22,904 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family2/7edd009624e64724bb94ccdd1b41c1b3 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family2/7edd009624e64724bb94ccdd1b41c1b3 2023-03-08 19:56:23,212 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testMajorCompactionFromAdmin' and column 'family1' 2023-03-08 19:56:23,236 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023030864d271fe41764bea9f399565e9001e78_del 2023-03-08 19:56:23,257 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230308e05246412da04adda61dfa47e61b1e20_del 2023-03-08 19:56:23,277 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202303088c307aba4efd41a799172c75c8c186e9_del 2023-03-08 19:56:23,277 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testMajorCompactionFromAdmin' column='family1' 2023-03-08 19:56:23,277 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] zookeeper.ReadOnlyZKClient(139): Connect 0x58840ef6 to 127.0.0.1:56751 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-08 19:56:23,282 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7765b67e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-08 19:56:23,282 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230308 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-08 19:56:24,237 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230308466515c725f6479d9f83881d248cd80e to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230308466515c725f6479d9f83881d248cd80e 2023-03-08 19:56:24,238 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230308 table='testMajorCompactionFromAdmin' column='family1' 2023-03-08 19:56:24,250 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-08 19:56:24,257 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35028, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-08 19:56:24,279 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-08 19:56:24,281 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48490, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-08 19:56:24,325 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230308/family1/284ccf9b1dca436fb63cddf66f13a083 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-03-08 19:56:24,326 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9., hostname=jenkins-hbase4.apache.org,33367,1678305218680, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230308/family1/284ccf9b1dca436fb63cddf66f13a083}] 2023-03-08 19:56:24,330 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(763): Validating hfile at hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230308/family1/284ccf9b1dca436fb63cddf66f13a083 for inclusion in 0abcf5c9f041342bc11db8ee9ff848c9/family1 2023-03-08 19:56:24,351 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-03-08 19:56:24,351 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-08 19:56:24,351 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(2446): Flush status journal for 0abcf5c9f041342bc11db8ee9ff848c9: 2023-03-08 19:56:24,352 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230308/family1/284ccf9b1dca436fb63cddf66f13a083 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionFromAdmin__hl60ah750odc3kn9e8q81mqcud8ekf9jv1c7v1c930o7r8kupk9gto4v8ph0r2g1/family1/284ccf9b1dca436fb63cddf66f13a083 2023-03-08 19:56:24,354 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionFromAdmin__hl60ah750odc3kn9e8q81mqcud8ekf9jv1c7v1c930o7r8kupk9gto4v8ph0r2g1/family1/284ccf9b1dca436fb63cddf66f13a083 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family1/9566f123608249249d4a2c35cff33be9_SeqId_28_ 2023-03-08 19:56:24,355 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(849): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionFromAdmin__hl60ah750odc3kn9e8q81mqcud8ekf9jv1c7v1c930o7r8kupk9gto4v8ph0r2g1/family1/284ccf9b1dca436fb63cddf66f13a083 into 0abcf5c9f041342bc11db8ee9ff848c9/family1 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family1/9566f123608249249d4a2c35cff33be9_SeqId_28_ - updating store file list. 2023-03-08 19:56:24,377 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(882): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family1/9566f123608249249d4a2c35cff33be9_SeqId_28_ into 0abcf5c9f041342bc11db8ee9ff848c9/family1 2023-03-08 19:56:24,377 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(855): Successfully loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionFromAdmin__hl60ah750odc3kn9e8q81mqcud8ekf9jv1c7v1c930o7r8kupk9gto4v8ph0r2g1/family1/284ccf9b1dca436fb63cddf66f13a083 into 0abcf5c9f041342bc11db8ee9ff848c9/family1 (new location: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/family1/9566f123608249249d4a2c35cff33be9_SeqId_28_) 2023-03-08 19:56:24,377 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionFromAdmin__hl60ah750odc3kn9e8q81mqcud8ekf9jv1c7v1c930o7r8kupk9gto4v8ph0r2g1/family1/284ccf9b1dca436fb63cddf66f13a083 2023-03-08 19:56:24,380 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230308 table='testMajorCompactionFromAdmin' column='family1' 2023-03-08 19:56:24,380 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:56:24,381 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023030814d3571b5cd246389fc73755dd60229a to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023030814d3571b5cd246389fc73755dd60229a 2023-03-08 19:56:24,382 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230308d5738920ce2447bcbde9ba5055ae6d78 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230308d5738920ce2447bcbde9ba5055ae6d78 2023-03-08 19:56:24,383 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230308e348a7514c2f4365ab01865dd88c95c1 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230308e348a7514c2f4365ab01865dd88c95c1 2023-03-08 19:56:24,384 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230308e8b5eadb54ef4028bf8e0819d2e4e30c to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230308e8b5eadb54ef4028bf8e0819d2e4e30c 2023-03-08 19:56:24,384 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-03-08 19:56:24,384 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230308' 2023-03-08 19:56:24,385 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230308 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-08 19:56:24,385 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230308 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-08 19:56:24,538 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303086146faae2cdd48fbbf8f6e2242acd195 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303086146faae2cdd48fbbf8f6e2242acd195 2023-03-08 19:56:24,539 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230308 table='testMajorCompactionFromAdmin' column='family1' 2023-03-08 19:56:24,587 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230308/family1/a37b8fbdb86f4fb1b4fc97168b735455 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-03-08 19:56:24,588 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811., hostname=jenkins-hbase4.apache.org,33367,1678305218680, seqNum=2 for row with hfile group [{family1,hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230308/family1/a37b8fbdb86f4fb1b4fc97168b735455}] 2023-03-08 19:56:24,590 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(763): Validating hfile at hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230308/family1/a37b8fbdb86f4fb1b4fc97168b735455 for inclusion in 78278c90009823fdb2b2cd7994a08811/family1 2023-03-08 19:56:24,611 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-03-08 19:56:24,611 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(777): Region bounds: first= last=1 2023-03-08 19:56:24,611 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(2446): Flush status journal for 78278c90009823fdb2b2cd7994a08811: 2023-03-08 19:56:24,612 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230308/family1/a37b8fbdb86f4fb1b4fc97168b735455 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionFromAdmin__k63ardpbvaf11gc909ifq96eboinb52bkfqttf744k77c183cohiv8valkh25lqr/family1/a37b8fbdb86f4fb1b4fc97168b735455 2023-03-08 19:56:24,615 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionFromAdmin__k63ardpbvaf11gc909ifq96eboinb52bkfqttf744k77c183cohiv8valkh25lqr/family1/a37b8fbdb86f4fb1b4fc97168b735455 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family1/e945e212ae9742f683630eef421406bb_SeqId_28_ 2023-03-08 19:56:24,615 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(849): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionFromAdmin__k63ardpbvaf11gc909ifq96eboinb52bkfqttf744k77c183cohiv8valkh25lqr/family1/a37b8fbdb86f4fb1b4fc97168b735455 into 78278c90009823fdb2b2cd7994a08811/family1 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family1/e945e212ae9742f683630eef421406bb_SeqId_28_ - updating store file list. 2023-03-08 19:56:24,644 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(882): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family1/e945e212ae9742f683630eef421406bb_SeqId_28_ into 78278c90009823fdb2b2cd7994a08811/family1 2023-03-08 19:56:24,644 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(855): Successfully loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionFromAdmin__k63ardpbvaf11gc909ifq96eboinb52bkfqttf744k77c183cohiv8valkh25lqr/family1/a37b8fbdb86f4fb1b4fc97168b735455 into 78278c90009823fdb2b2cd7994a08811/family1 (new location: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/family1/e945e212ae9742f683630eef421406bb_SeqId_28_) 2023-03-08 19:56:24,645 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionFromAdmin__k63ardpbvaf11gc909ifq96eboinb52bkfqttf744k77c183cohiv8valkh25lqr/family1/a37b8fbdb86f4fb1b4fc97168b735455 2023-03-08 19:56:24,647 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230308 table='testMajorCompactionFromAdmin' column='family1' 2023-03-08 19:56:24,648 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:56:24,649 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303085c3a777593b347e89dcc23570f0be7c8 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303085c3a777593b347e89dcc23570f0be7c8 2023-03-08 19:56:24,650 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303086c486dacc39b40ca869dfe3d685083a2 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303086c486dacc39b40ca869dfe3d685083a2 2023-03-08 19:56:24,651 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230308e4ba1f4ec9ce489683cb4c69cad942d8 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230308e4ba1f4ec9ce489683cb4c69cad942d8 2023-03-08 19:56:24,652 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230308f79f141dd7c84a5dad3f498a47157cc6 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230308f79f141dd7c84a5dad3f498a47157cc6 2023-03-08 19:56:24,652 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-03-08 19:56:24,652 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230308' 2023-03-08 19:56:24,652 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230308 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-08 19:56:24,652 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230308 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-08 19:56:25,198 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303088cad42c7af9f47a780384bdb444264ae to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303088cad42c7af9f47a780384bdb444264ae 2023-03-08 19:56:25,199 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230308 table='testMajorCompactionFromAdmin' column='family1' 2023-03-08 19:56:25,238 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230308/family1/bc7ff2df21654b8f896a551bc18eae28 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-03-08 19:56:25,239 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7., hostname=jenkins-hbase4.apache.org,33367,1678305218680, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230308/family1/bc7ff2df21654b8f896a551bc18eae28}] 2023-03-08 19:56:25,240 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(763): Validating hfile at hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230308/family1/bc7ff2df21654b8f896a551bc18eae28 for inclusion in 7becb875ae4303bd834b15dde14376f7/family1 2023-03-08 19:56:25,261 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-03-08 19:56:25,261 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-08 19:56:25,261 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegion(2446): Flush status journal for 7becb875ae4303bd834b15dde14376f7: 2023-03-08 19:56:25,261 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230308/family1/bc7ff2df21654b8f896a551bc18eae28 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionFromAdmin__24l08kja4u313pjaviguc669i86hrjctl33dd21ijeolmq2gus7mn134agdhltle/family1/bc7ff2df21654b8f896a551bc18eae28 2023-03-08 19:56:25,263 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionFromAdmin__24l08kja4u313pjaviguc669i86hrjctl33dd21ijeolmq2gus7mn134agdhltle/family1/bc7ff2df21654b8f896a551bc18eae28 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family1/f4f2e010a0ad46078d55af1a7df765a5_SeqId_28_ 2023-03-08 19:56:25,264 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(849): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionFromAdmin__24l08kja4u313pjaviguc669i86hrjctl33dd21ijeolmq2gus7mn134agdhltle/family1/bc7ff2df21654b8f896a551bc18eae28 into 7becb875ae4303bd834b15dde14376f7/family1 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family1/f4f2e010a0ad46078d55af1a7df765a5_SeqId_28_ - updating store file list. 2023-03-08 19:56:25,285 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(882): Loaded HFile hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family1/f4f2e010a0ad46078d55af1a7df765a5_SeqId_28_ into 7becb875ae4303bd834b15dde14376f7/family1 2023-03-08 19:56:25,285 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.HStore(855): Successfully loaded hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionFromAdmin__24l08kja4u313pjaviguc669i86hrjctl33dd21ijeolmq2gus7mn134agdhltle/family1/bc7ff2df21654b8f896a551bc18eae28 into 7becb875ae4303bd834b15dde14376f7/family1 (new location: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/family1/f4f2e010a0ad46078d55af1a7df765a5_SeqId_28_) 2023-03-08 19:56:25,285 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33367] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/staging/jenkins__testMajorCompactionFromAdmin__24l08kja4u313pjaviguc669i86hrjctl33dd21ijeolmq2gus7mn134agdhltle/family1/bc7ff2df21654b8f896a551bc18eae28 2023-03-08 19:56:25,287 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230308 table='testMajorCompactionFromAdmin' column='family1' 2023-03-08 19:56:25,287 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:56:25,288 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303085294a5359a0e474c839e398c59b5b936 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303085294a5359a0e474c839e398c59b5b936 2023-03-08 19:56:25,289 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303085cabfd51d72e4e5fb2676b8238f51919 to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303085cabfd51d72e4e5fb2676b8238f51919 2023-03-08 19:56:25,290 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303088f40467618774e95b811c224c93dba3d to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303088f40467618774e95b811c224c93dba3d 2023-03-08 19:56:25,291 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303089516075c89f64cd2aa0d3f06869b32cc to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303089516075c89f64cd2aa0d3f06869b32cc 2023-03-08 19:56:25,291 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-03-08 19:56:25,291 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230308' 2023-03-08 19:56:25,291 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230308 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-08 19:56:25,291 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testMajorCompactionFromAdmin' column='family1' 2023-03-08 19:56:25,291 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-08 19:56:25,291 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(367): [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023030864d271fe41764bea9f399565e9001e78_del] 2023-03-08 19:56:25,292 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:56:25,293 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023030864d271fe41764bea9f399565e9001e78_del to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023030864d271fe41764bea9f399565e9001e78_del 2023-03-08 19:56:25,293 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-03-08 19:56:25,293 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(367): [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230308e05246412da04adda61dfa47e61b1e20_del] 2023-03-08 19:56:25,293 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:56:25,294 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230308e05246412da04adda61dfa47e61b1e20_del to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230308e05246412da04adda61dfa47e61b1e20_del 2023-03-08 19:56:25,294 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-03-08 19:56:25,294 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] compactions.PartitionedMobCompactor(367): [hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303088c307aba4efd41a799172c75c8c186e9_del] 2023-03-08 19:56:25,295 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] backup.HFileArchiver(360): Archiving compacted files. 2023-03-08 19:56:25,295 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303088c307aba4efd41a799172c75c8c186e9_del to hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303088c307aba4efd41a799172c75c8c186e9_del 2023-03-08 19:56:25,295 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-03-08 19:56:25,296 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678305223354] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-08 19:56:25,297 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=123, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-03-08 19:56:25,298 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=123, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE in 2.4250 sec 2023-03-08 19:56:25,307 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-03-08 19:56:25,317 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 19:56:25,319 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 19:56:25,321 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-08 19:56:25,322 DEBUG [Listener at localhost/34049] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-08 19:56:25,325 INFO [Listener at localhost/34049] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-08 19:56:25,359 INFO [Listener at localhost/34049] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=248 (was 257), OpenFileDescriptor=640 (was 654), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=184 (was 187), ProcessCount=170 (was 170), AvailableMemoryMB=8163 (was 8184) 2023-03-08 19:56:25,360 DEBUG [Listener at localhost/34049] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x06140021 to 127.0.0.1:56751 2023-03-08 19:56:25,360 DEBUG [Listener at localhost/34049] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 19:56:25,360 INFO [Listener at localhost/34049] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-03-08 19:56:25,360 INFO [Listener at localhost/34049] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-08 19:56:25,360 DEBUG [Listener at localhost/34049] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6decc8b7 to 127.0.0.1:56751 2023-03-08 19:56:25,360 DEBUG [Listener at localhost/34049] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 19:56:25,360 DEBUG [Listener at localhost/34049] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-03-08 19:56:25,360 DEBUG [Listener at localhost/34049] util.JVMClusterUtil(257): Found active master hash=376350505, stopped=false 2023-03-08 19:56:25,360 INFO [Listener at localhost/34049] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,39989,1678305217508 2023-03-08 19:56:25,368 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-03-08 19:56:25,368 INFO [Listener at localhost/34049] procedure2.ProcedureExecutor(629): Stopping 2023-03-08 19:56:25,368 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-03-08 19:56:25,368 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-08 19:56:25,368 DEBUG [Listener at localhost/34049] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7337cf50 to 127.0.0.1:56751 2023-03-08 19:56:25,368 DEBUG [Listener at localhost/34049] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 19:56:25,369 INFO [Listener at localhost/34049] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,33367,1678305218680' ***** 2023-03-08 19:56:25,369 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-08 19:56:25,369 INFO [Listener at localhost/34049] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-03-08 19:56:25,369 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-08 19:56:25,369 INFO [RS:0;jenkins-hbase4:33367] regionserver.HeapMemoryManager(220): Stopping 2023-03-08 19:56:25,369 INFO [RS:0;jenkins-hbase4:33367] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-03-08 19:56:25,369 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-03-08 19:56:25,369 INFO [RS:0;jenkins-hbase4:33367] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-03-08 19:56:25,369 INFO [RS:0;jenkins-hbase4:33367] regionserver.HRegionServer(3304): Received CLOSE for 9c99479945efcf910b16288e607c83eb 2023-03-08 19:56:25,370 INFO [RS:0;jenkins-hbase4:33367] regionserver.HRegionServer(3304): Received CLOSE for 246f7af23020ff6b2e393b5a81b2e3c4 2023-03-08 19:56:25,370 INFO [RS:0;jenkins-hbase4:33367] regionserver.HRegionServer(3304): Received CLOSE for 0abcf5c9f041342bc11db8ee9ff848c9 2023-03-08 19:56:25,370 INFO [RS:0;jenkins-hbase4:33367] regionserver.HRegionServer(3304): Received CLOSE for 78278c90009823fdb2b2cd7994a08811 2023-03-08 19:56:25,370 INFO [RS:0;jenkins-hbase4:33367] regionserver.HRegionServer(3304): Received CLOSE for 400484c718b5dfcfabf2e62d3d3e1140 2023-03-08 19:56:25,370 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 9c99479945efcf910b16288e607c83eb, disabling compactions & flushes 2023-03-08 19:56:25,370 INFO [RS:0;jenkins-hbase4:33367] regionserver.HRegionServer(3304): Received CLOSE for c1edfa28825217aa75bd370b97cfc23d 2023-03-08 19:56:25,370 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. 2023-03-08 19:56:25,370 INFO [RS:0;jenkins-hbase4:33367] regionserver.HRegionServer(3304): Received CLOSE for 3e027aaacb7795251255fdd7284ec165 2023-03-08 19:56:25,370 INFO [RS:0;jenkins-hbase4:33367] regionserver.HRegionServer(3304): Received CLOSE for f43229ba0f524b69e8eb52fe67361236 2023-03-08 19:56:25,370 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. 2023-03-08 19:56:25,370 INFO [RS:0;jenkins-hbase4:33367] regionserver.HRegionServer(3304): Received CLOSE for 3551c80f1cc0fa597fb1a918e4b1b6f0 2023-03-08 19:56:25,370 INFO [RS:0;jenkins-hbase4:33367] regionserver.HRegionServer(3304): Received CLOSE for 2232a25e9ac1db961952c09e8e81f27b 2023-03-08 19:56:25,370 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. after waiting 0 ms 2023-03-08 19:56:25,370 INFO [RS:0;jenkins-hbase4:33367] regionserver.HRegionServer(3304): Received CLOSE for 7becb875ae4303bd834b15dde14376f7 2023-03-08 19:56:25,370 INFO [RS:0;jenkins-hbase4:33367] regionserver.HRegionServer(3304): Received CLOSE for 4ad1e88a375de73ef8abab9078c9a436 2023-03-08 19:56:25,370 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. 2023-03-08 19:56:25,370 INFO [RS:0;jenkins-hbase4:33367] regionserver.HRegionServer(3304): Received CLOSE for 2d54b0ef5ff21a425fa7a6fc1850c764 2023-03-08 19:56:25,370 INFO [RS:0;jenkins-hbase4:33367] regionserver.HRegionServer(3304): Received CLOSE for bf25ef5682cfde9aa90a5d6fe8229bdd 2023-03-08 19:56:25,370 INFO [RS:0;jenkins-hbase4:33367] regionserver.HRegionServer(3304): Received CLOSE for 4f3474af7bec67a38b1529f33c13c7a6 2023-03-08 19:56:25,370 INFO [RS:0;jenkins-hbase4:33367] regionserver.HRegionServer(3304): Received CLOSE for b4dc975db9245bfec2f0c0dc6879a14e 2023-03-08 19:56:25,370 INFO [RS:0;jenkins-hbase4:33367] regionserver.HRegionServer(3304): Received CLOSE for a526a5895c0e84ca75dee0352c5e6b73 2023-03-08 19:56:25,370 INFO [RS:0;jenkins-hbase4:33367] regionserver.HRegionServer(3304): Received CLOSE for d392b5a609f9661af565e4c73dab47f8 2023-03-08 19:56:25,370 INFO [RS:0;jenkins-hbase4:33367] regionserver.HRegionServer(3304): Received CLOSE for c0d4af7836e0c3ca2f373a671b92c8f6 2023-03-08 19:56:25,370 INFO [RS:0;jenkins-hbase4:33367] regionserver.HRegionServer(3304): Received CLOSE for b1a5a8585f5b1218e278f2f033438f28 2023-03-08 19:56:25,370 INFO [RS:0;jenkins-hbase4:33367] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:25,370 DEBUG [RS:0;jenkins-hbase4:33367] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1e1f4632 to 127.0.0.1:56751 2023-03-08 19:56:25,370 DEBUG [RS:0;jenkins-hbase4:33367] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 19:56:25,371 INFO [RS:0;jenkins-hbase4:33367] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-03-08 19:56:25,371 INFO [RS:0;jenkins-hbase4:33367] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-03-08 19:56:25,371 INFO [RS:0;jenkins-hbase4:33367] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-03-08 19:56:25,371 INFO [RS:0;jenkins-hbase4:33367] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-03-08 19:56:25,371 INFO [RS:0;jenkins-hbase4:33367] regionserver.HRegionServer(1475): Waiting on 21 regions to close 2023-03-08 19:56:25,371 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-03-08 19:56:25,371 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-03-08 19:56:25,371 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-03-08 19:56:25,371 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-03-08 19:56:25,372 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-03-08 19:56:25,372 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=50.56 KB heapSize=78.23 KB 2023-03-08 19:56:25,372 DEBUG [RS:0;jenkins-hbase4:33367] regionserver.HRegionServer(1505): Waiting on 0abcf5c9f041342bc11db8ee9ff848c9, 1588230740, 2232a25e9ac1db961952c09e8e81f27b, 246f7af23020ff6b2e393b5a81b2e3c4, 2d54b0ef5ff21a425fa7a6fc1850c764, 3551c80f1cc0fa597fb1a918e4b1b6f0, 3e027aaacb7795251255fdd7284ec165, 400484c718b5dfcfabf2e62d3d3e1140, 4ad1e88a375de73ef8abab9078c9a436, 4f3474af7bec67a38b1529f33c13c7a6, 78278c90009823fdb2b2cd7994a08811, 7becb875ae4303bd834b15dde14376f7, 9c99479945efcf910b16288e607c83eb, a526a5895c0e84ca75dee0352c5e6b73, b1a5a8585f5b1218e278f2f033438f28, b4dc975db9245bfec2f0c0dc6879a14e, bf25ef5682cfde9aa90a5d6fe8229bdd, c0d4af7836e0c3ca2f373a671b92c8f6, c1edfa28825217aa75bd370b97cfc23d, d392b5a609f9661af565e4c73dab47f8, f43229ba0f524b69e8eb52fe67361236 2023-03-08 19:56:25,375 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/9c99479945efcf910b16288e607c83eb/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-03-08 19:56:25,376 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. 2023-03-08 19:56:25,376 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 9c99479945efcf910b16288e607c83eb: 2023-03-08 19:56:25,376 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,1,1678305311601.9c99479945efcf910b16288e607c83eb. 2023-03-08 19:56:25,376 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 246f7af23020ff6b2e393b5a81b2e3c4, disabling compactions & flushes 2023-03-08 19:56:25,376 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4. 2023-03-08 19:56:25,376 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4. 2023-03-08 19:56:25,376 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4. after waiting 0 ms 2023-03-08 19:56:25,376 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4. 2023-03-08 19:56:25,388 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/246f7af23020ff6b2e393b5a81b2e3c4/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-08 19:56:25,389 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4. 2023-03-08 19:56:25,389 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 246f7af23020ff6b2e393b5a81b2e3c4: 2023-03-08 19:56:25,389 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,,1678305338013.246f7af23020ff6b2e393b5a81b2e3c4. 2023-03-08 19:56:25,389 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 0abcf5c9f041342bc11db8ee9ff848c9, disabling compactions & flushes 2023-03-08 19:56:25,389 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9. 2023-03-08 19:56:25,389 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9. 2023-03-08 19:56:25,389 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9. after waiting 0 ms 2023-03-08 19:56:25,389 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9. 2023-03-08 19:56:25,392 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/0abcf5c9f041342bc11db8ee9ff848c9/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-08 19:56:25,392 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9. 2023-03-08 19:56:25,392 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 0abcf5c9f041342bc11db8ee9ff848c9: 2023-03-08 19:56:25,393 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,2,1678305365537.0abcf5c9f041342bc11db8ee9ff848c9. 2023-03-08 19:56:25,393 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 78278c90009823fdb2b2cd7994a08811, disabling compactions & flushes 2023-03-08 19:56:25,393 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811. 2023-03-08 19:56:25,393 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811. 2023-03-08 19:56:25,393 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811. after waiting 0 ms 2023-03-08 19:56:25,393 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811. 2023-03-08 19:56:25,395 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/78278c90009823fdb2b2cd7994a08811/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-08 19:56:25,396 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811. 2023-03-08 19:56:25,396 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 78278c90009823fdb2b2cd7994a08811: 2023-03-08 19:56:25,396 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,,1678305365537.78278c90009823fdb2b2cd7994a08811. 2023-03-08 19:56:25,396 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 400484c718b5dfcfabf2e62d3d3e1140, disabling compactions & flushes 2023-03-08 19:56:25,396 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140. 2023-03-08 19:56:25,396 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140. 2023-03-08 19:56:25,396 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140. after waiting 0 ms 2023-03-08 19:56:25,396 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140. 2023-03-08 19:56:25,399 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/400484c718b5dfcfabf2e62d3d3e1140/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-08 19:56:25,399 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140. 2023-03-08 19:56:25,399 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 400484c718b5dfcfabf2e62d3d3e1140: 2023-03-08 19:56:25,400 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,1,1678305338013.400484c718b5dfcfabf2e62d3d3e1140. 2023-03-08 19:56:25,400 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing c1edfa28825217aa75bd370b97cfc23d, disabling compactions & flushes 2023-03-08 19:56:25,400 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:56:25,400 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:56:25,400 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. after waiting 0 ms 2023-03-08 19:56:25,400 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:56:25,402 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/c1edfa28825217aa75bd370b97cfc23d/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-03-08 19:56:25,403 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:56:25,403 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for c1edfa28825217aa75bd370b97cfc23d: 2023-03-08 19:56:25,403 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678305275723.c1edfa28825217aa75bd370b97cfc23d. 2023-03-08 19:56:25,403 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 3e027aaacb7795251255fdd7284ec165, disabling compactions & flushes 2023-03-08 19:56:25,403 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1678305221812.3e027aaacb7795251255fdd7284ec165. 2023-03-08 19:56:25,403 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1678305221812.3e027aaacb7795251255fdd7284ec165. 2023-03-08 19:56:25,403 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1678305221812.3e027aaacb7795251255fdd7284ec165. after waiting 0 ms 2023-03-08 19:56:25,403 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1678305221812.3e027aaacb7795251255fdd7284ec165. 2023-03-08 19:56:25,403 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 3e027aaacb7795251255fdd7284ec165 1/1 column families, dataSize=109 B heapSize=592 B 2023-03-08 19:56:25,404 INFO [regionserver/jenkins-hbase4:0.Chore.2] hbase.ScheduledChore(146): Chore: CompactedHFilesCleaner was stopped 2023-03-08 19:56:25,460 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-03-08 19:56:25,572 DEBUG [RS:0;jenkins-hbase4:33367] regionserver.HRegionServer(1505): Waiting on 1588230740, 2232a25e9ac1db961952c09e8e81f27b, 2d54b0ef5ff21a425fa7a6fc1850c764, 3551c80f1cc0fa597fb1a918e4b1b6f0, 3e027aaacb7795251255fdd7284ec165, 4ad1e88a375de73ef8abab9078c9a436, 4f3474af7bec67a38b1529f33c13c7a6, 7becb875ae4303bd834b15dde14376f7, a526a5895c0e84ca75dee0352c5e6b73, b1a5a8585f5b1218e278f2f033438f28, b4dc975db9245bfec2f0c0dc6879a14e, bf25ef5682cfde9aa90a5d6fe8229bdd, c0d4af7836e0c3ca2f373a671b92c8f6, d392b5a609f9661af565e4c73dab47f8, f43229ba0f524b69e8eb52fe67361236 2023-03-08 19:56:25,772 DEBUG [RS:0;jenkins-hbase4:33367] regionserver.HRegionServer(1505): Waiting on 1588230740, 2232a25e9ac1db961952c09e8e81f27b, 2d54b0ef5ff21a425fa7a6fc1850c764, 3551c80f1cc0fa597fb1a918e4b1b6f0, 3e027aaacb7795251255fdd7284ec165, 4ad1e88a375de73ef8abab9078c9a436, 4f3474af7bec67a38b1529f33c13c7a6, 7becb875ae4303bd834b15dde14376f7, a526a5895c0e84ca75dee0352c5e6b73, b1a5a8585f5b1218e278f2f033438f28, b4dc975db9245bfec2f0c0dc6879a14e, bf25ef5682cfde9aa90a5d6fe8229bdd, c0d4af7836e0c3ca2f373a671b92c8f6, d392b5a609f9661af565e4c73dab47f8, f43229ba0f524b69e8eb52fe67361236 2023-03-08 19:56:25,788 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=48.28 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/hbase/meta/1588230740/.tmp/info/df3c5954da5c4468b93693d448477139 2023-03-08 19:56:25,793 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for df3c5954da5c4468b93693d448477139 2023-03-08 19:56:25,801 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=102 B at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/hbase/meta/1588230740/.tmp/rep_barrier/f4a9a4b3c6c44a7e83c6c17bd5959086 2023-03-08 19:56:25,804 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f4a9a4b3c6c44a7e83c6c17bd5959086 2023-03-08 19:56:25,811 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=109 B at sequenceid=7 (bloomFilter=true), to=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/hbase/namespace/3e027aaacb7795251255fdd7284ec165/.tmp/info/1c5990f580584f80856aeefe7c920c06 2023-03-08 19:56:25,816 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/hbase/namespace/3e027aaacb7795251255fdd7284ec165/.tmp/info/1c5990f580584f80856aeefe7c920c06 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/hbase/namespace/3e027aaacb7795251255fdd7284ec165/info/1c5990f580584f80856aeefe7c920c06 2023-03-08 19:56:25,819 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/hbase/namespace/3e027aaacb7795251255fdd7284ec165/info/1c5990f580584f80856aeefe7c920c06, entries=3, sequenceid=7, filesize=4.8 K 2023-03-08 19:56:25,820 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~109 B/109, heapSize ~576 B/576, currentSize=0 B/0 for 3e027aaacb7795251255fdd7284ec165 in 417ms, sequenceid=7, compaction requested=false 2023-03-08 19:56:25,824 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/hbase/namespace/3e027aaacb7795251255fdd7284ec165/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-03-08 19:56:25,824 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1678305221812.3e027aaacb7795251255fdd7284ec165. 2023-03-08 19:56:25,825 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 3e027aaacb7795251255fdd7284ec165: 2023-03-08 19:56:25,825 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1678305221812.3e027aaacb7795251255fdd7284ec165. 2023-03-08 19:56:25,825 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing f43229ba0f524b69e8eb52fe67361236, disabling compactions & flushes 2023-03-08 19:56:25,825 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,2,1678305258777.f43229ba0f524b69e8eb52fe67361236. 2023-03-08 19:56:25,825 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1678305258777.f43229ba0f524b69e8eb52fe67361236. 2023-03-08 19:56:25,825 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1678305258777.f43229ba0f524b69e8eb52fe67361236. after waiting 0 ms 2023-03-08 19:56:25,825 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1678305258777.f43229ba0f524b69e8eb52fe67361236. 2023-03-08 19:56:25,827 INFO [regionserver/jenkins-hbase4:0.Chore.2] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-03-08 19:56:25,827 INFO [regionserver/jenkins-hbase4:0.Chore.2] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-03-08 19:56:25,829 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/f43229ba0f524b69e8eb52fe67361236/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-03-08 19:56:25,829 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,2,1678305258777.f43229ba0f524b69e8eb52fe67361236. 2023-03-08 19:56:25,829 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for f43229ba0f524b69e8eb52fe67361236: 2023-03-08 19:56:25,830 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,2,1678305258777.f43229ba0f524b69e8eb52fe67361236. 2023-03-08 19:56:25,830 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 3551c80f1cc0fa597fb1a918e4b1b6f0, disabling compactions & flushes 2023-03-08 19:56:25,830 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. 2023-03-08 19:56:25,830 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. 2023-03-08 19:56:25,830 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. after waiting 0 ms 2023-03-08 19:56:25,830 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. 2023-03-08 19:56:25,832 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithMonthlyPolicy/3551c80f1cc0fa597fb1a918e4b1b6f0/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-03-08 19:56:25,833 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. 2023-03-08 19:56:25,833 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 3551c80f1cc0fa597fb1a918e4b1b6f0: 2023-03-08 19:56:25,833 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithMonthlyPolicy,,1678305236294.3551c80f1cc0fa597fb1a918e4b1b6f0. 2023-03-08 19:56:25,833 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2232a25e9ac1db961952c09e8e81f27b, disabling compactions & flushes 2023-03-08 19:56:25,833 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,1,1678305258777.2232a25e9ac1db961952c09e8e81f27b. 2023-03-08 19:56:25,833 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1678305258777.2232a25e9ac1db961952c09e8e81f27b. 2023-03-08 19:56:25,833 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1678305258777.2232a25e9ac1db961952c09e8e81f27b. after waiting 0 ms 2023-03-08 19:56:25,833 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1678305258777.2232a25e9ac1db961952c09e8e81f27b. 2023-03-08 19:56:25,835 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/2232a25e9ac1db961952c09e8e81f27b/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-03-08 19:56:25,836 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,1,1678305258777.2232a25e9ac1db961952c09e8e81f27b. 2023-03-08 19:56:25,836 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2232a25e9ac1db961952c09e8e81f27b: 2023-03-08 19:56:25,836 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,1,1678305258777.2232a25e9ac1db961952c09e8e81f27b. 2023-03-08 19:56:25,836 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 7becb875ae4303bd834b15dde14376f7, disabling compactions & flushes 2023-03-08 19:56:25,836 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7. 2023-03-08 19:56:25,836 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7. 2023-03-08 19:56:25,836 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7. after waiting 0 ms 2023-03-08 19:56:25,836 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7. 2023-03-08 19:56:25,839 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionFromAdmin/7becb875ae4303bd834b15dde14376f7/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-08 19:56:25,840 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7. 2023-03-08 19:56:25,840 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 7becb875ae4303bd834b15dde14376f7: 2023-03-08 19:56:25,840 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,1,1678305365537.7becb875ae4303bd834b15dde14376f7. 2023-03-08 19:56:25,840 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 4ad1e88a375de73ef8abab9078c9a436, disabling compactions & flushes 2023-03-08 19:56:25,840 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:56:25,840 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:56:25,840 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. after waiting 0 ms 2023-03-08 19:56:25,840 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:56:25,843 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/4ad1e88a375de73ef8abab9078c9a436/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-03-08 19:56:25,843 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:56:25,843 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 4ad1e88a375de73ef8abab9078c9a436: 2023-03-08 19:56:25,843 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1678305223532.4ad1e88a375de73ef8abab9078c9a436. 2023-03-08 19:56:25,843 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2d54b0ef5ff21a425fa7a6fc1850c764, disabling compactions & flushes 2023-03-08 19:56:25,843 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. 2023-03-08 19:56:25,844 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. 2023-03-08 19:56:25,844 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. after waiting 0 ms 2023-03-08 19:56:25,844 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. 2023-03-08 19:56:25,846 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyPolicy/2d54b0ef5ff21a425fa7a6fc1850c764/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-08 19:56:25,847 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. 2023-03-08 19:56:25,847 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2d54b0ef5ff21a425fa7a6fc1850c764: 2023-03-08 19:56:25,847 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyPolicy,,1678305262792.2d54b0ef5ff21a425fa7a6fc1850c764. 2023-03-08 19:56:25,847 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing bf25ef5682cfde9aa90a5d6fe8229bdd, disabling compactions & flushes 2023-03-08 19:56:25,847 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. 2023-03-08 19:56:25,847 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. 2023-03-08 19:56:25,847 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. after waiting 0 ms 2023-03-08 19:56:25,847 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. 2023-03-08 19:56:25,850 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/bf25ef5682cfde9aa90a5d6fe8229bdd/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-03-08 19:56:25,851 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. 2023-03-08 19:56:25,851 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for bf25ef5682cfde9aa90a5d6fe8229bdd: 2023-03-08 19:56:25,851 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,,1678305311601.bf25ef5682cfde9aa90a5d6fe8229bdd. 2023-03-08 19:56:25,851 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 4f3474af7bec67a38b1529f33c13c7a6, disabling compactions & flushes 2023-03-08 19:56:25,851 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6. 2023-03-08 19:56:25,851 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6. 2023-03-08 19:56:25,851 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6. after waiting 0 ms 2023-03-08 19:56:25,851 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6. 2023-03-08 19:56:25,854 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/ns/testMinorCompaction/4f3474af7bec67a38b1529f33c13c7a6/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-08 19:56:25,854 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6. 2023-03-08 19:56:25,854 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 4f3474af7bec67a38b1529f33c13c7a6: 2023-03-08 19:56:25,854 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,2,1678305338013.4f3474af7bec67a38b1529f33c13c7a6. 2023-03-08 19:56:25,854 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b4dc975db9245bfec2f0c0dc6879a14e, disabling compactions & flushes 2023-03-08 19:56:25,854 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,,1678305258777.b4dc975db9245bfec2f0c0dc6879a14e. 2023-03-08 19:56:25,854 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1678305258777.b4dc975db9245bfec2f0c0dc6879a14e. 2023-03-08 19:56:25,855 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1678305258777.b4dc975db9245bfec2f0c0dc6879a14e. after waiting 0 ms 2023-03-08 19:56:25,855 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1678305258777.b4dc975db9245bfec2f0c0dc6879a14e. 2023-03-08 19:56:25,857 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testScannerOnBulkLoadRefHFiles/b4dc975db9245bfec2f0c0dc6879a14e/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-03-08 19:56:25,857 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,,1678305258777.b4dc975db9245bfec2f0c0dc6879a14e. 2023-03-08 19:56:25,857 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b4dc975db9245bfec2f0c0dc6879a14e: 2023-03-08 19:56:25,857 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,,1678305258777.b4dc975db9245bfec2f0c0dc6879a14e. 2023-03-08 19:56:25,857 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a526a5895c0e84ca75dee0352c5e6b73, disabling compactions & flushes 2023-03-08 19:56:25,857 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:56:25,857 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:56:25,857 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. after waiting 0 ms 2023-03-08 19:56:25,857 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:56:25,860 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/a526a5895c0e84ca75dee0352c5e6b73/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-03-08 19:56:25,860 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:56:25,860 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a526a5895c0e84ca75dee0352c5e6b73: 2023-03-08 19:56:25,860 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678305291095.a526a5895c0e84ca75dee0352c5e6b73. 2023-03-08 19:56:25,860 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d392b5a609f9661af565e4c73dab47f8, disabling compactions & flushes 2023-03-08 19:56:25,860 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. 2023-03-08 19:56:25,860 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. 2023-03-08 19:56:25,861 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. after waiting 0 ms 2023-03-08 19:56:25,861 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. 2023-03-08 19:56:25,863 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testCompactionWithHFileLink/d392b5a609f9661af565e4c73dab47f8/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-03-08 19:56:25,864 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. 2023-03-08 19:56:25,864 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d392b5a609f9661af565e4c73dab47f8: 2023-03-08 19:56:25,864 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,2,1678305311601.d392b5a609f9661af565e4c73dab47f8. 2023-03-08 19:56:25,864 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing c0d4af7836e0c3ca2f373a671b92c8f6, disabling compactions & flushes 2023-03-08 19:56:25,864 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. 2023-03-08 19:56:25,864 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. 2023-03-08 19:56:25,864 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. after waiting 0 ms 2023-03-08 19:56:25,864 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. 2023-03-08 19:56:25,867 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithWeeklyPolicy/c0d4af7836e0c3ca2f373a671b92c8f6/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-08 19:56:25,867 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. 2023-03-08 19:56:25,867 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for c0d4af7836e0c3ca2f373a671b92c8f6: 2023-03-08 19:56:25,867 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithWeeklyPolicy,,1678305248716.c0d4af7836e0c3ca2f373a671b92c8f6. 2023-03-08 19:56:25,867 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b1a5a8585f5b1218e278f2f033438f28, disabling compactions & flushes 2023-03-08 19:56:25,867 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. 2023-03-08 19:56:25,867 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. 2023-03-08 19:56:25,867 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. after waiting 0 ms 2023-03-08 19:56:25,867 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. 2023-03-08 19:56:25,879 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/default/testMinorCompactionWithMonthlyPolicy/b1a5a8585f5b1218e278f2f033438f28/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-03-08 19:56:25,879 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. 2023-03-08 19:56:25,879 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b1a5a8585f5b1218e278f2f033438f28: 2023-03-08 19:56:25,879 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithMonthlyPolicy,,1678305355929.b1a5a8585f5b1218e278f2f033438f28. 2023-03-08 19:56:25,973 DEBUG [RS:0;jenkins-hbase4:33367] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-03-08 19:56:26,151 DEBUG [regionserver/jenkins-hbase4:0.Chore.2] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-03-08 19:56:26,173 DEBUG [RS:0;jenkins-hbase4:33367] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-03-08 19:56:26,213 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.18 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/hbase/meta/1588230740/.tmp/table/994ceb6c06d1449ba1629e297de7cd57 2023-03-08 19:56:26,219 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 994ceb6c06d1449ba1629e297de7cd57 2023-03-08 19:56:26,219 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/hbase/meta/1588230740/.tmp/info/df3c5954da5c4468b93693d448477139 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/hbase/meta/1588230740/info/df3c5954da5c4468b93693d448477139 2023-03-08 19:56:26,224 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for df3c5954da5c4468b93693d448477139 2023-03-08 19:56:26,224 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/hbase/meta/1588230740/info/df3c5954da5c4468b93693d448477139, entries=226, sequenceid=130, filesize=37.9 K 2023-03-08 19:56:26,225 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/hbase/meta/1588230740/.tmp/rep_barrier/f4a9a4b3c6c44a7e83c6c17bd5959086 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/hbase/meta/1588230740/rep_barrier/f4a9a4b3c6c44a7e83c6c17bd5959086 2023-03-08 19:56:26,228 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f4a9a4b3c6c44a7e83c6c17bd5959086 2023-03-08 19:56:26,228 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/hbase/meta/1588230740/rep_barrier/f4a9a4b3c6c44a7e83c6c17bd5959086, entries=1, sequenceid=130, filesize=5.0 K 2023-03-08 19:56:26,229 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/hbase/meta/1588230740/.tmp/table/994ceb6c06d1449ba1629e297de7cd57 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/hbase/meta/1588230740/table/994ceb6c06d1449ba1629e297de7cd57 2023-03-08 19:56:26,232 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 994ceb6c06d1449ba1629e297de7cd57 2023-03-08 19:56:26,232 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/hbase/meta/1588230740/table/994ceb6c06d1449ba1629e297de7cd57, entries=27, sequenceid=130, filesize=6.6 K 2023-03-08 19:56:26,234 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~50.56 KB/51772, heapSize ~78.19 KB/80064, currentSize=0 B/0 for 1588230740 in 862ms, sequenceid=130, compaction requested=false 2023-03-08 19:56:26,237 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/data/hbase/meta/1588230740/recovered.edits/133.seqid, newMaxSeqId=133, maxSeqId=1 2023-03-08 19:56:26,237 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-03-08 19:56:26,237 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-03-08 19:56:26,237 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-03-08 19:56:26,237 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-03-08 19:56:26,373 INFO [RS:0;jenkins-hbase4:33367] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,33367,1678305218680; all regions closed. 2023-03-08 19:56:26,381 DEBUG [RS:0;jenkins-hbase4:33367] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/oldWALs 2023-03-08 19:56:26,381 INFO [RS:0;jenkins-hbase4:33367] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C33367%2C1678305218680.meta:.meta(num 1678305221552) 2023-03-08 19:56:26,385 DEBUG [RS:0;jenkins-hbase4:33367] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/oldWALs 2023-03-08 19:56:26,385 INFO [RS:0;jenkins-hbase4:33367] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C33367%2C1678305218680:(num 1678305220966) 2023-03-08 19:56:26,385 DEBUG [RS:0;jenkins-hbase4:33367] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 19:56:26,385 INFO [RS:0;jenkins-hbase4:33367] regionserver.LeaseManager(133): Closed leases 2023-03-08 19:56:26,386 INFO [RS:0;jenkins-hbase4:33367] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-03-08 19:56:26,386 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-03-08 19:56:26,387 INFO [RS:0;jenkins-hbase4:33367] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:33367 2023-03-08 19:56:26,395 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,33367,1678305218680 2023-03-08 19:56:26,395 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-03-08 19:56:26,395 ERROR [Listener at localhost/34049-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@2c0674d3 rejected from java.util.concurrent.ThreadPoolExecutor@2839f58b[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 531] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-08 19:56:26,396 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-03-08 19:56:26,396 ERROR [Listener at localhost/34049-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@43a303f2 rejected from java.util.concurrent.ThreadPoolExecutor@2839f58b[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 531] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-08 19:56:26,397 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,33367,1678305218680] 2023-03-08 19:56:26,397 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,33367,1678305218680; numProcessing=1 2023-03-08 19:56:26,399 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,33367,1678305218680 already deleted, retry=false 2023-03-08 19:56:26,399 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,33367,1678305218680 expired; onlineServers=0 2023-03-08 19:56:26,399 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,39989,1678305217508' ***** 2023-03-08 19:56:26,399 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-03-08 19:56:26,399 DEBUG [M:0;jenkins-hbase4:39989] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5753ad12, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-03-08 19:56:26,399 INFO [M:0;jenkins-hbase4:39989] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,39989,1678305217508 2023-03-08 19:56:26,399 INFO [M:0;jenkins-hbase4:39989] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,39989,1678305217508; all regions closed. 2023-03-08 19:56:26,399 DEBUG [M:0;jenkins-hbase4:39989] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-08 19:56:26,400 DEBUG [M:0;jenkins-hbase4:39989] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-03-08 19:56:26,400 DEBUG [M:0;jenkins-hbase4:39989] cleaner.HFileCleaner(317): Stopping file delete threads 2023-03-08 19:56:26,400 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-03-08 19:56:26,400 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1678305220683] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1678305220683,5,FailOnTimeoutGroup] 2023-03-08 19:56:26,400 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1678305220684] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1678305220684,5,FailOnTimeoutGroup] 2023-03-08 19:56:26,400 INFO [M:0;jenkins-hbase4:39989] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-03-08 19:56:26,400 INFO [M:0;jenkins-hbase4:39989] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-03-08 19:56:26,400 INFO [M:0;jenkins-hbase4:39989] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-03-08 19:56:26,401 DEBUG [M:0;jenkins-hbase4:39989] master.HMaster(1512): Stopping service threads 2023-03-08 19:56:26,401 INFO [M:0;jenkins-hbase4:39989] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-03-08 19:56:26,401 ERROR [M:0;jenkins-hbase4:39989] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-20,5,PEWorkerGroup] Thread[HFileArchiver-21,5,PEWorkerGroup] Thread[HFileArchiver-22,5,PEWorkerGroup] 2023-03-08 19:56:26,402 INFO [M:0;jenkins-hbase4:39989] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-03-08 19:56:26,402 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-03-08 19:56:26,405 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-03-08 19:56:26,405 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-08 19:56:26,405 DEBUG [M:0;jenkins-hbase4:39989] zookeeper.ZKUtil(398): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-03-08 19:56:26,405 WARN [M:0;jenkins-hbase4:39989] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-03-08 19:56:26,405 INFO [M:0;jenkins-hbase4:39989] assignment.AssignmentManager(315): Stopping assignment manager 2023-03-08 19:56:26,405 INFO [M:0;jenkins-hbase4:39989] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-03-08 19:56:26,405 DEBUG [M:0;jenkins-hbase4:39989] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-03-08 19:56:26,405 INFO [M:0;jenkins-hbase4:39989] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-08 19:56:26,405 DEBUG [M:0;jenkins-hbase4:39989] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-08 19:56:26,405 DEBUG [M:0;jenkins-hbase4:39989] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-03-08 19:56:26,406 DEBUG [M:0;jenkins-hbase4:39989] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-08 19:56:26,406 INFO [M:0;jenkins-hbase4:39989] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=512.95 KB heapSize=599.13 KB 2023-03-08 19:56:26,407 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-03-08 19:56:26,414 INFO [M:0;jenkins-hbase4:39989] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=512.95 KB at sequenceid=1011 (bloomFilter=true), to=hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/12f19d471d6f475e8e6497c3fbf0a019 2023-03-08 19:56:26,417 INFO [M:0;jenkins-hbase4:39989] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 12f19d471d6f475e8e6497c3fbf0a019 2023-03-08 19:56:26,418 DEBUG [M:0;jenkins-hbase4:39989] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/12f19d471d6f475e8e6497c3fbf0a019 as hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/12f19d471d6f475e8e6497c3fbf0a019 2023-03-08 19:56:26,421 INFO [M:0;jenkins-hbase4:39989] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 12f19d471d6f475e8e6497c3fbf0a019 2023-03-08 19:56:26,421 INFO [M:0;jenkins-hbase4:39989] regionserver.HStore(1080): Added hdfs://localhost:42127/user/jenkins/test-data/a40bd9b6-29de-95be-12c8-a657998cb488/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/12f19d471d6f475e8e6497c3fbf0a019, entries=123, sequenceid=1011, filesize=33.3 K 2023-03-08 19:56:26,422 INFO [M:0;jenkins-hbase4:39989] regionserver.HRegion(2948): Finished flush of dataSize ~512.95 KB/525261, heapSize ~599.11 KB/613488, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 16ms, sequenceid=1011, compaction requested=false 2023-03-08 19:56:26,422 INFO [M:0;jenkins-hbase4:39989] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-08 19:56:26,422 DEBUG [M:0;jenkins-hbase4:39989] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-03-08 19:56:26,425 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-03-08 19:56:26,425 INFO [M:0;jenkins-hbase4:39989] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-03-08 19:56:26,425 INFO [M:0;jenkins-hbase4:39989] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:39989 2023-03-08 19:56:26,428 DEBUG [M:0;jenkins-hbase4:39989] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,39989,1678305217508 already deleted, retry=false 2023-03-08 19:56:26,497 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-08 19:56:26,497 INFO [RS:0;jenkins-hbase4:33367] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,33367,1678305218680; zookeeper connection closed. 2023-03-08 19:56:26,497 ERROR [Listener at localhost/34049-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@3b6e741c rejected from java.util.concurrent.ThreadPoolExecutor@2839f58b[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 531] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-08 19:56:26,498 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): regionserver:33367-0x100c5b1324f0001, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-08 19:56:26,498 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@5c26d8f4] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@5c26d8f4 2023-03-08 19:56:26,498 ERROR [Listener at localhost/34049-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5673bf64 rejected from java.util.concurrent.ThreadPoolExecutor@2839f58b[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 531] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-08 19:56:26,498 INFO [Listener at localhost/34049] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-03-08 19:56:26,597 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-08 19:56:26,597 INFO [M:0;jenkins-hbase4:39989] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,39989,1678305217508; zookeeper connection closed. 2023-03-08 19:56:26,598 ERROR [Listener at localhost/34049-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@367517bf rejected from java.util.concurrent.ThreadPoolExecutor@2254351a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 906] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-08 19:56:26,598 DEBUG [Listener at localhost/34049-EventThread] zookeeper.ZKWatcher(600): master:39989-0x100c5b1324f0000, quorum=127.0.0.1:56751, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-08 19:56:26,598 ERROR [Listener at localhost/34049-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@24f9c261 rejected from java.util.concurrent.ThreadPoolExecutor@2254351a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 906] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-08 19:56:26,599 WARN [Listener at localhost/34049] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-03-08 19:56:26,602 INFO [Listener at localhost/34049] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-03-08 19:56:26,708 WARN [BP-715752202-172.31.14.131-1678305214181 heartbeating to localhost/127.0.0.1:42127] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-03-08 19:56:26,708 WARN [BP-715752202-172.31.14.131-1678305214181 heartbeating to localhost/127.0.0.1:42127] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-715752202-172.31.14.131-1678305214181 (Datanode Uuid cf18edd4-f033-4bf5-a039-714536ab9194) service to localhost/127.0.0.1:42127 2023-03-08 19:56:26,710 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/36526ac9-e9a0-8ea5-8063-2f26c591d24b/cluster_63dc3982-03ab-365c-777c-42eb5a69ef67/dfs/data/data1/current/BP-715752202-172.31.14.131-1678305214181] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-03-08 19:56:26,710 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/36526ac9-e9a0-8ea5-8063-2f26c591d24b/cluster_63dc3982-03ab-365c-777c-42eb5a69ef67/dfs/data/data2/current/BP-715752202-172.31.14.131-1678305214181] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-03-08 19:56:26,735 INFO [Listener at localhost/34049] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-03-08 19:56:26,873 INFO [Listener at localhost/34049] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-03-08 19:56:26,957 INFO [Listener at localhost/34049] hbase.HBaseTestingUtility(1293): Minicluster is down